-- Logs begin at Thu 2022-07-28 21:06:06 UTC, end at Thu 2022-07-28 22:45:47 UTC. -- Jul 28 22:13:49.556462 dv-cinder-850939-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 28 22:13:49.563980 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 28 22:13:49.568165 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 28 22:13:49 2022] *** Jul 28 22:13:49.568165 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 28 22:13:49.568165 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 28 22:13:49.568165 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: nodename: dv-cinder-850939-3-iscsi Jul 28 22:13:49.568165 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: machine: x86_64 Jul 28 22:13:49.568165 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: clock source: unix Jul 28 22:13:49.568165 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: pcre jit disabled Jul 28 22:13:49.568165 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: detected number of CPU cores: 4 Jul 28 22:13:49.568165 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: current working directory: / Jul 28 22:13:49.568165 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: detected binary path: /usr/bin/uwsgi-core Jul 28 22:13:49.568165 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: your processes number limit is 47680 Jul 28 22:13:49.568165 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: your memory page size is 4096 bytes Jul 28 22:13:49.568165 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: detected max file descriptor number: 2048 Jul 28 22:13:49.568165 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: lock engine: pthread robust mutexes Jul 28 22:13:49.568165 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: thunder lock: enabled Jul 28 22:13:49.568165 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 28 22:13:49.568165 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 28 22:13:49.606481 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: Python main interpreter initialized at 0x55d6237ec9c0 Jul 28 22:13:49.606481 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: python threads support enabled Jul 28 22:13:49.606481 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: your server socket listen backlog is limited to 100 connections Jul 28 22:13:49.606481 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: your mercy for graceful operations on workers is 90 seconds Jul 28 22:13:49.606842 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: mapped 403077 bytes (393 KB) for 2 cores Jul 28 22:13:49.606914 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: *** Operational MODE: preforking *** Jul 28 22:13:49.607166 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: *** uWSGI is running in multiple interpreter mode *** Jul 28 22:13:49.607166 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: spawned uWSGI master process (pid: 80356) Jul 28 22:13:49.607368 dv-cinder-850939-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 28 22:13:49.609354 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: spawned uWSGI worker 1 (pid: 80362, cores: 1) Jul 28 22:13:49.609354 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: spawned uWSGI worker 2 (pid: 80363, cores: 1) Jul 28 22:13:49.609354 dv-cinder-850939-3-iscsi devstack@keystone.service[80356]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 28 22:13:55.166681 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 22:13:55.166681 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: from cryptography.utils import int_from_bytes Jul 28 22:13:55.167403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 22:13:55.167403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: from cryptography.utils import int_from_bytes Jul 28 22:13:55.167536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 22:13:55.167536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: from cryptography.utils import int_from_bytes Jul 28 22:13:55.167536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 28 22:13:55.167536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: from cryptography.utils import int_from_bytes Jul 28 22:13:55.216424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 22:13:55.217314 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 22:13:55.217424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 28 22:13:55.217664 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] command line args: [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 22:13:55.217776 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 28 22:13:55.217856 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] command line args: [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 28 22:13:55.218182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 22:13:55.218278 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 28 22:13:55.218278 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ================================================================================ {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 22:13:55.218420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ================================================================================ {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 28 22:13:55.218504 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] admin_token = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.218671 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] admin_token = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.218759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] config_dir = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.218920 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] config_dir = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.219006 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.219085 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] config_source = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.219169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.219273 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] debug = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.219361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] config_source = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.219434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] debug = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.219513 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.219790 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.219919 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.220024 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.220122 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.220229 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.220333 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.220430 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.220532 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.220634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.220746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.220856 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.220959 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_config_append = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.221062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_config_append = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.221158 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.221266 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.221369 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_dir = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.221476 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_dir = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.221650 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_file = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.221773 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_file = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.221861 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.221989 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.222072 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.222153 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.222271 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.222353 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.222427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.222553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.222632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.222759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.222845 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.222972 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.223050 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.223167 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.223256 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.223337 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.223416 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.223533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.223609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.223724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.223812 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.223892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.223964 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.224080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.224172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.224284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.224377 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.224532 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.224634 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.224811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.225112 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.225293 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] publish_errors = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.225378 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.225484 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] publish_errors = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.225607 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.225744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.225829 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.225926 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.226003 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.226150 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.226238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.226384 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.226462 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.226614 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] standard_threads = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.226695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.226878 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.226958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] standard_threads = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.227081 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.227160 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.227233 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.227309 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.227434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] use_journal = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.227512 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.227590 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] use_json = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.227663 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] use_journal = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.227740 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] use_stderr = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.227826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] use_json = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.227901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] use_syslog = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.227977 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] use_stderr = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.228083 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.228169 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] use_syslog = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.228333 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.228423 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 28 22:13:55.228568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.228652 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.228814 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.228897 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.229047 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.229135 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.229291 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.229370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.229554 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.229637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.229792 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.229881 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.230025 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.230109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.230268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.230346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.230510 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.230596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.230757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.230836 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.230990 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.231075 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.231232 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.231317 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.231463 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.231555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.231721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.231809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.231950 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.232105 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.232218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.232303 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.232545 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.232951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.233038 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.233131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.233206 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.233283 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.233357 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.233441 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.233605 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.233703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.233866 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.233952 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.234116 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.234195 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.234363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.234450 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.234593 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.234672 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.234822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.234901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.235051 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.235138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.235283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.235361 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.235523 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.235601 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.235765 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.235844 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.235994 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.236073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.236221 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.236305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.236459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.236538 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.236700 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.236778 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.236935 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.237013 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.237193 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.237321 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.237405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.237479 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.237660 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.237749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.237910 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.237996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.238110 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.238191 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.238338 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.238418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.238566 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.238646 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.238794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.238874 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.239029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.239108 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.239267 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.239347 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.239493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.239579 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.239730 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.239810 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.240089 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.240177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.240240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.240306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.240421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.240503 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.240660 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.240740 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.240887 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.240983 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.241127 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.241264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.241351 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.241427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.241606 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.241696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.241843 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.241927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.242080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.242169 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.242307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.242386 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.242540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.242626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.242774 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.242860 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.243012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.243109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.243242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.243331 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.243475 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.243560 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.243720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.243801 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.243949 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.244030 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.244186 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.244265 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.244423 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.244506 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.244663 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.244758 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.244907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.244983 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.245154 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.245237 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.245398 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.245518 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.245655 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.245749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.245890 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.245971 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.246124 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.246204 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.246363 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.246455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.246598 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.246683 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.246833 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.246923 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.247074 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.247162 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.247306 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.247392 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.247544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.247626 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.247777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.247866 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.248040 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.248111 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.248347 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.248437 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.248547 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.248637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.248787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.248875 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.249028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.249126 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.249265 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.249423 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.249501 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.249598 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.249754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.249844 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.249990 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.250080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.250224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.250320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.250452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.250533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.250681 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.250760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.250910 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.250999 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.251143 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.251225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.251373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.251469 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.251608 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.251700 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.251841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.251948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.252085 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.252165 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.252318 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.252412 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.252566 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.252645 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.252820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.252907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.253061 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.253151 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.253291 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.external = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.253379 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.253519 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.253656 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.external = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.253778 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.253859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.254053 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.254143 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.254283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.password = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.254364 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.254512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.token = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.254594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.password = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.254744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] auth.token = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.254824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.254974 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.255054 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.255213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.255301 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.255439 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.255521 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.255682 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.255764 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.255925 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.256006 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.256155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.256239 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.256389 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.256470 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.256625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.256706 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.256869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.256951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.257114 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.257195 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.257354 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.257435 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.257614 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.257698 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.257850 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.257930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.258091 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.258172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.258346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.258428 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.258582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.258664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.258832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.258913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.259087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.259169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.259342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.259421 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.259603 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.259684 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.259845 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.259929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.260097 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.260185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.260332 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.260414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.260568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.260651 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.260807 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.260893 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.261049 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.261130 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.261278 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.261362 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.261517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.261613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.261780 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.261907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.262021 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.262087 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.262262 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.262355 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.262510 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.262591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.262771 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.262912 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.263042 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.263170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.263297 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.263957 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.264168 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.264253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.264317 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.264377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.264521 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.264675 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.264803 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.264945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.265074 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.265201 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.265329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.265469 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.265655 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.266044 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.266172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.266301 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.266427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.266571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.266702 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.266829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.266970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.267099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.267226 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.267362 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.267497 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.267627 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.267755 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.267882 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.268011 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.268152 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.268281 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.268410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.268537 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.268665 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.268810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.268955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.269086 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.269215 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.269343 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.269470 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.269676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.269770 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.269847 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.269921 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.269994 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.270067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.270141 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.270222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.270298 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.270371 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.270444 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.270517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.270589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.270674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.270749 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.270823 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.270903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.270977 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.271050 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.271131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.271212 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.271367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.271450 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.271609 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.271691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.271861 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.271944 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.272095 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.272177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.272327 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.272408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.272564 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.272646 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.272796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.272875 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.273034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.273114 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.273271 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.273402 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.273507 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.273618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.273770 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.273850 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.274012 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.274096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.274240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.274321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.274469 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.274549 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.274715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.274798 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.274946 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.275026 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.275179 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.275258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.275413 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.275495 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.275645 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.275727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.275875 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.275957 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.276112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.276194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.276350 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.276431 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.276584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.276672 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.276828 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.276909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.277067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.277153 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.277307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.277400 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.277580 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.277666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.277795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.277876 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.278026 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.278120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.278258 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.278340 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.278494 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.278575 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.278724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.278814 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.278962 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.279042 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.279808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.279945 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.280094 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.280212 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.280435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.280520 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.280681 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.280763 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.280911 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.281069 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.281152 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.281327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.281409 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.281585 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.281670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.281843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.281926 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.282079 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.282205 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.282321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.282402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.282564 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.282646 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.282799 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.282880 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.283048 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.283130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.283274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.283355 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.283515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.283596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.283749 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.283831 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.283982 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.284065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.284217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.284298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.284457 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.284537 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.284695 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.284776 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.284923 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.285005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.285157 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.285237 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.285395 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.285477 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.285650 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.285732 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.285890 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.285971 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.286124 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.286206 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.286371 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.286451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.286618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.286699 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.286852 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.286936 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.287096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.287177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.287331 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.287412 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.287560 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.287639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.287788 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] role.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.287869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.288039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.288119 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] role.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.288264 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.288346 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.288507 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] role.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.288587 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.288749 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.288830 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] role.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.288983 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.289065 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.289222 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.289303 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.289463 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.289565 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.289732 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.289816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.289975 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.290057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.290217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.290307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.290448 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.290530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.290692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.290772 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.290936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.291026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.291172 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.291293 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.291413 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.291493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.291661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.291811 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.291915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.292047 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.292166 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.292294 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.292529 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.292622 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.292775 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.292856 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.293023 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.293105 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.293256 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.293352 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.293509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.293622 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.293784 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.293865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.294018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.294112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.294248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.294330 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.294476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.294558 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.294750 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.294844 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.295072 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.295221 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.295329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.295487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.295587 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.295695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.295800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.295954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.296036 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.296217 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.296307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.296469 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.296549 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.296704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.296788 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.296942 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.297030 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.297174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.297254 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.297411 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.297491 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.297678 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.297774 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.297911 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.297992 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.298155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.298236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.298395 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.298486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.298639 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.298720 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.298875 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.298955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.299108 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.299197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.299337 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.299419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.299578 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.299659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.299819 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.299908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.300044 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.300132 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.300282 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.300362 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.300529 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.300619 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.300781 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.300862 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.connection = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.301013 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.301104 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.301257 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.301339 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.301491 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.301606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.301754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.301836 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.connection = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.301988 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.302079 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.302222 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.302303 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.302456 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.302538 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.302689 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.302770 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.302930 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.303011 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.303165 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.303247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.303405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.303486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.303644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.303759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.303886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.303979 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.304119 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.304201 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.304360 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.304441 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.304587 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.304670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.304823 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.304904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.305055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.305136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.305218 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 22:13:55.305295 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.305536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.305787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.306025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 28 22:13:55.306199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 28 22:13:55.334589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.334816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.335036 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.335507 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.335745 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.335977 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.343984 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.345038 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.374047 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.374286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.375357 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.375594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.426777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.427439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.477549 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.477794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.478584 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.478825 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.546075 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.546307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.546535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.546648 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.546788 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.546870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.546982 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.547068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.547182 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.547268 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.547428 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.547641 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.570769 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.570893 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.570996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.571111 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.571225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.571334 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.571453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.571536 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.571677 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.571760 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.571902 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.571984 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.572121 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.572205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.583997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.584604 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.596766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.597003 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.597219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.597415 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.597501 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.597671 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.597870 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.598018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.598162 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.598304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.598447 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.598447 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.598721 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.598721 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 28 22:13:55.615277 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.615485 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 22:13:55.615682 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.615816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.615902 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 22:13:55.616014 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 22:13:55.616218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.616301 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.616420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 28 22:13:55.616511 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.616692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.616774 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.616956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.617036 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.617218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.617474 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.627154 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.627428 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.627695 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.627778 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.627968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.628053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.628238 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.628523 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.634849 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.635483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.640136 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.640413 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.640689 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.641228 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.641333 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.641502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.641608 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.641777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.641860 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.642038 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.642133 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.642305 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.642389 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.642567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.642649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.642833 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.642916 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.643092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.643174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.643367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.643643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.643911 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.665915 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.666899 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.667385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.667667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.668399 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.668681 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.672267 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.672543 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.672817 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.673563 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.673846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.674132 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.679414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.679648 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.680772 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.681056 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.683796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.684079 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.684357 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.684643 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.684915 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.685192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.685273 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.686087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.686178 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.686440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.686537 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.686663 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.686939 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.687225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.687496 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.687772 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.703051 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.703323 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.704320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.704603 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.707242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.708533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.710385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.710675 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.711724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.712021 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.715050 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.716437 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.718340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.719916 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.721454 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.721773 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.722063 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.722352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.722632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.722919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.723122 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.723422 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.723709 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.723997 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.724279 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.724572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.739415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.739685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.739946 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.740203 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.741134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.741423 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.741722 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.741993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.747080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.749215 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.749481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.749594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.751064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.751342 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.752296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.752572 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.752859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.753152 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.753435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.754206 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.754491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.754782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.755082 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.755369 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.764835 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.765109 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.765389 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.766021 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.766310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.766581 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.766862 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.767138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.767422 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.767705 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.767985 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.768264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.768538 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.768814 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.782532 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.784483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.785446 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.787499 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.788377 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.790475 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.790846 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.792352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.792630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.792904 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.793031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.794587 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.795396 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.795677 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.800669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.802259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.802849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.804752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.805490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.805803 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.806084 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.806354 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.807865 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.808159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.808470 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.808759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.815348 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.815635 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.815986 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.818725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.819307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.820209 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.823171 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.823529 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.823983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.824269 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.824530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.824863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.825173 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.825451 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.825880 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.826966 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.827279 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.827634 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.827857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.828001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.828176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 28 22:13:55.828462 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.828747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.829161 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.829382 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.829788 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.830197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.830571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.831381 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.831381 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.831703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.831893 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.832179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.832467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 28 22:13:55.865259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d6237ec9c0 pid: 80363 (default app) Jul 28 22:13:55.868708 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5413a-2405-44d3-b0b9-de0aa5e9a60d None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:13:55.868998 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5413a-2405-44d3-b0b9-de0aa5e9a60d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:13:55.869273 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5413a-2405-44d3-b0b9-de0aa5e9a60d None None] PATH_INFO: `/v3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:13:55.871112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55d6237ec9c0 pid: 80362 (default app) Jul 28 22:13:55.874300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1/1] 10.222.0.39 () {50 vars in 791 bytes} [Thu Jul 28 22:13:55 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:03.606372 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46e10d-7911-45a0-99f7-8d679d59d1d7 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:03.606961 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46e10d-7911-45a0-99f7-8d679d59d1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:03.607124 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d46e10d-7911-45a0-99f7-8d679d59d1d7 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:03.611501 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1/2] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:03 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:14:03.616649 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb238d-0e57-4d23-96a0-c68244fee6d8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:03.617034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb238d-0e57-4d23-96a0-c68244fee6d8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:03.617197 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deb238d-0e57-4d23-96a0-c68244fee6d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:03.688376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: INFO dbcounter [None req-4deb238d-0e57-4d23-96a0-c68244fee6d8 None None] Registered counter for database keystone Jul 28 22:14:03.709790 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG oslo_db.sqlalchemy.engines [None req-4deb238d-0e57-4d23-96a0-c68244fee6d8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80363) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 22:14:03.715821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writer thread running {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 22:14:03.901867 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG passlib.handlers.bcrypt [None req-4deb238d-0e57-4d23-96a0-c68244fee6d8 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80363) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 22:14:03.902191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG passlib.handlers.bcrypt [None req-4deb238d-0e57-4d23-96a0-c68244fee6d8 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80363) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 22:14:03.956288 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4deb238d-0e57-4d23-96a0-c68244fee6d8 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:03.957920 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4deb238d-0e57-4d23-96a0-c68244fee6d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:04.088748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 2/3] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 22:14:04.219072 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46e10d-7911-45a0-99f7-8d679d59d1d7 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:04.288214 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: INFO dbcounter [None req-9d46e10d-7911-45a0-99f7-8d679d59d1d7 None None] Registered counter for database keystone Jul 28 22:14:04.307783 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG oslo_db.sqlalchemy.engines [None req-9d46e10d-7911-45a0-99f7-8d679d59d1d7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80362) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 28 22:14:04.314844 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writer thread running {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 28 22:14:04.437952 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d46e10d-7911-45a0-99f7-8d679d59d1d7 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:04.442081 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9df43-52a3-4a86-893a-8b72c5b8d7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:04.442707 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9df43-52a3-4a86-893a-8b72c5b8d7bb admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:04.442988 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9df43-52a3-4a86-893a-8b72c5b8d7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:04.443258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9df43-52a3-4a86-893a-8b72c5b8d7bb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:04.453732 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9df43-52a3-4a86-893a-8b72c5b8d7bb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:04.505870 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a7f9df43-52a3-4a86-893a-8b72c5b8d7bb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 22:14:04.506221 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9df43-52a3-4a86-893a-8b72c5b8d7bb admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:04.516171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 2/4] 10.222.0.39 () {58 vars in 1324 bytes} [Thu Jul 28 22:14:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 22:14:04.522182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb238d-0e57-4d23-96a0-c68244fee6d8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:04.537126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deb238d-0e57-4d23-96a0-c68244fee6d8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:04.541452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb12078-2197-45bd-a632-3b11fe455781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:04.542583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb12078-2197-45bd-a632-3b11fe455781 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:04.542867 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb12078-2197-45bd-a632-3b11fe455781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:04.543142 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb12078-2197-45bd-a632-3b11fe455781 admin admin] PATH_INFO: `/v3/projects/26d9130b7556481bb766f1b06a63e1e5` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:04.545215 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb12078-2197-45bd-a632-3b11fe455781 admin admin] RBAC: Authorizing `identity:get_project(project_id=26d9130b7556481bb766f1b06a63e1e5)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:04.595312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4bb12078-2197-45bd-a632-3b11fe455781 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 28 22:14:04.595655 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb12078-2197-45bd-a632-3b11fe455781 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:04.600287 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 3/5] 10.222.0.39 () {56 vars in 1207 bytes} [Thu Jul 28 22:14:04 2022] GET /identity/v3/projects/26d9130b7556481bb766f1b06a63e1e5 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:06.587996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3ff68-0b76-44cd-9002-3fa08f87d109 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:06.588495 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3ff68-0b76-44cd-9002-3fa08f87d109 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:06.588495 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b3ff68-0b76-44cd-9002-3fa08f87d109 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:06.593229 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 3/6] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:06.598581 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df3e0d-81d0-4eac-858a-729874411abb None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:06.598963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df3e0d-81d0-4eac-858a-729874411abb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:06.599134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df3e0d-81d0-4eac-858a-729874411abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:06.618657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-23df3e0d-81d0-4eac-858a-729874411abb None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:06.620172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-23df3e0d-81d0-4eac-858a-729874411abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:06.628980 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 4/7] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 28 22:14:06.651566 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3ff68-0b76-44cd-9002-3fa08f87d109 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:06.663566 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b3ff68-0b76-44cd-9002-3fa08f87d109 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:06.667779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b675e65-97ac-4e91-88e8-5c7f072fa613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:06.668383 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b675e65-97ac-4e91-88e8-5c7f072fa613 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:06.668666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b675e65-97ac-4e91-88e8-5c7f072fa613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:06.668945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b675e65-97ac-4e91-88e8-5c7f072fa613 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:06.676667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b675e65-97ac-4e91-88e8-5c7f072fa613 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:06.678465 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b675e65-97ac-4e91-88e8-5c7f072fa613 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:06.687924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 4/8] 10.222.0.39 () {58 vars in 1324 bytes} [Thu Jul 28 22:14:06 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 22:14:06.693000 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df3e0d-81d0-4eac-858a-729874411abb None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:06.702622 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df3e0d-81d0-4eac-858a-729874411abb None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:06.706989 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641a6a3-cad1-45d9-938a-8a3ecaffe79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:06.707614 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641a6a3-cad1-45d9-938a-8a3ecaffe79b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:06.707896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641a6a3-cad1-45d9-938a-8a3ecaffe79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:06.708168 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641a6a3-cad1-45d9-938a-8a3ecaffe79b admin admin] PATH_INFO: `/v3/users/0b20c2888f9642efb7858178880ed9a0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:06.709517 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641a6a3-cad1-45d9-938a-8a3ecaffe79b admin admin] RBAC: Authorizing `identity:get_user(user_id=0b20c2888f9642efb7858178880ed9a0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:06.711379 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641a6a3-cad1-45d9-938a-8a3ecaffe79b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:06.715597 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 5/9] 10.222.0.39 () {56 vars in 1198 bytes} [Thu Jul 28 22:14:06 2022] GET /identity/v3/users/0b20c2888f9642efb7858178880ed9a0 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:08.761246 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617f2ad-27fc-431f-b45c-5f4ad89e05b4 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:08.761808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617f2ad-27fc-431f-b45c-5f4ad89e05b4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:08.761893 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f617f2ad-27fc-431f-b45c-5f4ad89e05b4 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:08.763742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 5/10] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:08.769210 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13fa45-2b0d-4b59-a034-e67dc290cb3e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:08.769633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13fa45-2b0d-4b59-a034-e67dc290cb3e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:08.769915 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13fa45-2b0d-4b59-a034-e67dc290cb3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:08.789474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0a13fa45-2b0d-4b59-a034-e67dc290cb3e None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:08.790701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0a13fa45-2b0d-4b59-a034-e67dc290cb3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:08.852156 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e3ff7-9265-40c1-a854-74509ccf6b83 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:08.852439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e3ff7-9265-40c1-a854-74509ccf6b83 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:08.852718 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41e3ff7-9265-40c1-a854-74509ccf6b83 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:08.854365 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 6/11] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:14:08.859311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec6dd6-6d23-4a4f-b10a-507dfc481c36 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:08.859595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec6dd6-6d23-4a4f-b10a-507dfc481c36 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:08.859868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec6dd6-6d23-4a4f-b10a-507dfc481c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:08.869861 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 6/12] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:08.893659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13fa45-2b0d-4b59-a034-e67dc290cb3e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:08.902835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG passlib.handlers.bcrypt [None req-c0ec6dd6-6d23-4a4f-b10a-507dfc481c36 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80362) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 28 22:14:08.903266 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG passlib.handlers.bcrypt [None req-c0ec6dd6-6d23-4a4f-b10a-507dfc481c36 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80362) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 28 22:14:08.916946 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:08.948934 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c0ec6dd6-6d23-4a4f-b10a-507dfc481c36 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:08.951342 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c0ec6dd6-6d23-4a4f-b10a-507dfc481c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:08.998312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13fa45-2b0d-4b59-a034-e67dc290cb3e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:09.032252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d882c1-d39b-430b-8033-9e0374b63b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:09.032822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d882c1-d39b-430b-8033-9e0374b63b9b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:09.033105 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d882c1-d39b-430b-8033-9e0374b63b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:09.033378 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d882c1-d39b-430b-8033-9e0374b63b9b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:09.044599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d882c1-d39b-430b-8033-9e0374b63b9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:09.050539 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 7/13] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:09.053393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-05d882c1-d39b-430b-8033-9e0374b63b9b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:14:09.053393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:09.053393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:09.053393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:09.053393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:09.053393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:09.053393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:09.053393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:09.053393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:09.053393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:09.053393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:09.053393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:09.053393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:09.053393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:09.053393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:09.053393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:09.054344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:09.054344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:09.054344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:09.054344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:09.054344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:09.054344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:09.054344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:09.054344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:09.054344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:09.054344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:09.054344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:09.054344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:09.054344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:09.054344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:09.054344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:09.054344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:09.055187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:09.055187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:09.055187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:09.055187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:09.055187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:09.055187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:09.055187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:09.055187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:09.055187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:09.055187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:09.055187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:14:09.055187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:09.055187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 7/14] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:14:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:14:09.062277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec6dd6-6d23-4a4f-b10a-507dfc481c36 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:09.072554 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d882c1-d39b-430b-8033-9e0374b63b9b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:09.157476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:09.166568 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec6dd6-6d23-4a4f-b10a-507dfc481c36 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:09.169399 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:09.176869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d882c1-d39b-430b-8033-9e0374b63b9b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:09.204763 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ab8c3-47ba-4923-9b94-5c6378d7cdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:09.205668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ab8c3-47ba-4923-9b94-5c6378d7cdbd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:09.205952 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ab8c3-47ba-4923-9b94-5c6378d7cdbd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:09.206228 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ab8c3-47ba-4923-9b94-5c6378d7cdbd None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:09.207114 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ab8c3-47ba-4923-9b94-5c6378d7cdbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:09.207424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ab8c3-47ba-4923-9b94-5c6378d7cdbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:09.208856 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ab8c3-47ba-4923-9b94-5c6378d7cdbd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:09.211799 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52efef4-fd0a-4732-a769-a39c1d8d119b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:09.212423 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52efef4-fd0a-4732-a769-a39c1d8d119b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:09.212704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52efef4-fd0a-4732-a769-a39c1d8d119b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:09.212981 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52efef4-fd0a-4732-a769-a39c1d8d119b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:09.220575 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 8/15] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:09.226335 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ab8c3-47ba-4923-9b94-5c6378d7cdbd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:09.282106 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52efef4-fd0a-4732-a769-a39c1d8d119b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:09.283639 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52efef4-fd0a-4732-a769-a39c1d8d119b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:09.327569 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ab8c3-47ba-4923-9b94-5c6378d7cdbd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:09.361861 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffc6eb-df09-4311-982d-0f92d5a6784c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:09.362712 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffc6eb-df09-4311-982d-0f92d5a6784c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:09.363087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffc6eb-df09-4311-982d-0f92d5a6784c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:09.363373 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ffc6eb-df09-4311-982d-0f92d5a6784c None admin] PATH_INFO: `/v3/users/0b20c2888f9642efb7858178880ed9a0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:09.363765 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:09.365301 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffc6eb-df09-4311-982d-0f92d5a6784c None admin] RBAC: Authorizing `identity:get_user(user_id=0b20c2888f9642efb7858178880ed9a0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:09.366965 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ffc6eb-df09-4311-982d-0f92d5a6784c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:09.368763 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 9/16] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:09 2022] GET /identity/v3/users/0b20c2888f9642efb7858178880ed9a0 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:09.374521 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffc6eb-df09-4311-982d-0f92d5a6784c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:09.384893 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 8/17] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 22:14:09.390199 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52efef4-fd0a-4732-a769-a39c1d8d119b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:09.410765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:09.482128 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ffc6eb-df09-4311-982d-0f92d5a6784c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:09.518761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a3de7-5ff7-4752-a25d-78b6f2009159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:09.519421 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a3de7-5ff7-4752-a25d-78b6f2009159 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:09.519709 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a3de7-5ff7-4752-a25d-78b6f2009159 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:09.519990 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2a3de7-5ff7-4752-a25d-78b6f2009159 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:09.521185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a3de7-5ff7-4752-a25d-78b6f2009159 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:09.522699 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2a3de7-5ff7-4752-a25d-78b6f2009159 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:09.524328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 10/18] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:14:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:09.530372 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a3de7-5ff7-4752-a25d-78b6f2009159 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:09.594498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52efef4-fd0a-4732-a769-a39c1d8d119b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:09.628016 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790da35b-fbec-43c7-96dd-ade62de4b86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:09.628665 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790da35b-fbec-43c7-96dd-ade62de4b86e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:09.628955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790da35b-fbec-43c7-96dd-ade62de4b86e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:09.629231 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790da35b-fbec-43c7-96dd-ade62de4b86e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:09.630686 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2a3de7-5ff7-4752-a25d-78b6f2009159 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:09.632173 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:09.640074 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790da35b-fbec-43c7-96dd-ade62de4b86e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:09.641828 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790da35b-fbec-43c7-96dd-ade62de4b86e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:09.657260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-790da35b-fbec-43c7-96dd-ade62de4b86e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:14:09.657260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:09.657260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:09.657260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:09.657260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:09.657260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:09.657260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:09.657260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:09.657260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:09.657260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:09.657260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:09.657260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:09.657260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:14:09.657260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:14:09.657260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:14:09.658237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:14:09.658237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:09.658237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:09.658237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:09.658237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:09.658237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:09.658237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:09.658237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:09.658237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:09.658237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:09.658237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:09.658237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:09.658237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:09.658237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:09.658237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:09.658237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:09.658237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:09.659045 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:14:09.659045 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:14:09.659045 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:14:09.659045 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:09.659045 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 9/19] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:14:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:14:09.664514 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790da35b-fbec-43c7-96dd-ade62de4b86e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:09.666142 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9cbf4-b086-4ce2-8cbb-1ac23af962d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:09.666760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9cbf4-b086-4ce2-8cbb-1ac23af962d6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:09.667042 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9cbf4-b086-4ce2-8cbb-1ac23af962d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:09.667316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9cbf4-b086-4ce2-8cbb-1ac23af962d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:09.668110 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9cbf4-b086-4ce2-8cbb-1ac23af962d6 None admin] RBAC: Adding query filter params (role.id=aded7bd43e974686b3c23bd9d4255b24, scope.domain.id=default, user.id=0b20c2888f9642efb7858178880ed9a0) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:09.668423 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9cbf4-b086-4ce2-8cbb-1ac23af962d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:09.670682 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9cbf4-b086-4ce2-8cbb-1ac23af962d6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:09.679428 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 11/20] 10.222.0.39 () {56 vars in 1322 bytes} [Thu Jul 28 22:14:09 2022] GET /identity/v3/role_assignments?user.id=0b20c2888f9642efb7858178880ed9a0&scope.domain.id=default&role.id=aded7bd43e974686b3c23bd9d4255b24 => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:09.702731 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:09.777127 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790da35b-fbec-43c7-96dd-ade62de4b86e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:09.815298 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f62353-6ad3-4fe2-b656-e645986b9bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:09.815936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f62353-6ad3-4fe2-b656-e645986b9bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:09.816222 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f62353-6ad3-4fe2-b656-e645986b9bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:09.816500 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f62353-6ad3-4fe2-b656-e645986b9bd2 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:09.817211 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f62353-6ad3-4fe2-b656-e645986b9bd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:09.817518 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f62353-6ad3-4fe2-b656-e645986b9bd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:09.818958 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f62353-6ad3-4fe2-b656-e645986b9bd2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:09.837485 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 10/21] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:12.084483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d3ef5-0cf4-450b-94f5-ad13cb1d7422 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:12.085338 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d3ef5-0cf4-450b-94f5-ad13cb1d7422 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:12.085887 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d3ef5-0cf4-450b-94f5-ad13cb1d7422 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:12.087395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 12/22] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:12.089004 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8979f-1177-4ab6-98fc-1cfaf4e29df0 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:12.089801 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8979f-1177-4ab6-98fc-1cfaf4e29df0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:12.090252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8979f-1177-4ab6-98fc-1cfaf4e29df0 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:12.092429 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 11/23] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:12.093760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03136d33-3202-471e-aef1-a521659f4c04 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:12.094253 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03136d33-3202-471e-aef1-a521659f4c04 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:12.094725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03136d33-3202-471e-aef1-a521659f4c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:12.099442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b287a2-6aad-45eb-9a06-48d64557eea5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:12.100399 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b287a2-6aad-45eb-9a06-48d64557eea5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:12.100887 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b287a2-6aad-45eb-9a06-48d64557eea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:12.134632 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-03136d33-3202-471e-aef1-a521659f4c04 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:12.136341 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-03136d33-3202-471e-aef1-a521659f4c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:12.142541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-56b287a2-6aad-45eb-9a06-48d64557eea5 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:12.144812 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-56b287a2-6aad-45eb-9a06-48d64557eea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:12.303009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 13/24] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:12.303844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 12/25] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:12.354217 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03136d33-3202-471e-aef1-a521659f4c04 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:12.354767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b287a2-6aad-45eb-9a06-48d64557eea5 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:12.398957 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:12.505920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:12.510644 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03136d33-3202-471e-aef1-a521659f4c04 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:12.549337 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b287a2-6aad-45eb-9a06-48d64557eea5 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:12.575055 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c40d41-b66f-4f44-9bb4-be250b75d180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:12.576494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c40d41-b66f-4f44-9bb4-be250b75d180 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:12.576932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c40d41-b66f-4f44-9bb4-be250b75d180 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:12.577218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c40d41-b66f-4f44-9bb4-be250b75d180 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:12.600118 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c40d41-b66f-4f44-9bb4-be250b75d180 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:12.611872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-15c40d41-b66f-4f44-9bb4-be250b75d180 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:14:12.611872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:12.611872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:12.611872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:12.611872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:12.611872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:12.611872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:12.611872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:12.611872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:12.611872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:12.611872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:12.611872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:12.611872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:12.611872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:12.611872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:12.611872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:12.612743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:12.612743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:12.612743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:12.612743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:12.612743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:12.612743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:12.612743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:12.612743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:12.612743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:12.612743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:12.612743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:12.612743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:12.612743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:12.612743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:12.612743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:12.612743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:12.613497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:12.613497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:12.613497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:12.613497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:12.613497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:12.613497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:12.613497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:12.613497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:12.613497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:12.613497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:12.613497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:14:12.613497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:14:12.613497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 14/26] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:14:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:12.614966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a7df8-919f-425f-b256-f285ad32e16f None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:12.614966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a7df8-919f-425f-b256-f285ad32e16f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:12.614966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a7df8-919f-425f-b256-f285ad32e16f None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:12.620134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 15/27] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:12.620936 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a7df8-919f-425f-b256-f285ad32e16f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:12.636317 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121dd9fa-f32c-4906-92a0-fa2673bf67a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:12.636974 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dd9fa-f32c-4906-92a0-fa2673bf67a6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:12.637253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dd9fa-f32c-4906-92a0-fa2673bf67a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:12.637542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121dd9fa-f32c-4906-92a0-fa2673bf67a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:12.655259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121dd9fa-f32c-4906-92a0-fa2673bf67a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:12.656746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121dd9fa-f32c-4906-92a0-fa2673bf67a6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:12.678712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-121dd9fa-f32c-4906-92a0-fa2673bf67a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:14:12.678712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:12.678712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:12.678712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:12.678712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:12.678712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:12.678712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:12.678712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:12.678712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:12.678712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:12.678712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:12.678712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:12.678712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:14:12.678712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:14:12.678712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:14:12.679577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:14:12.679577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:12.679577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:12.679577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:12.679577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:12.679577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:12.679577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:12.679577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:12.679577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:12.679577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:12.679577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:12.679577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:12.679577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:12.679577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:12.679577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:12.679577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:12.679577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:12.680375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:14:12.680375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:14:12.680375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:14:12.680375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:12.680375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 13/28] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:14:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:14:12.681068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57921be-e18c-4cd1-9c67-d9d2096447c6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:12.681516 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57921be-e18c-4cd1-9c67-d9d2096447c6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:12.681976 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57921be-e18c-4cd1-9c67-d9d2096447c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:12.702687 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f57921be-e18c-4cd1-9c67-d9d2096447c6 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:12.704097 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f57921be-e18c-4cd1-9c67-d9d2096447c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:12.770996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a7df8-919f-425f-b256-f285ad32e16f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:12.798208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 14/29] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:12.800736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9d7dd-2dbc-48ed-b04c-6a7c8f07841e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:12.801167 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9d7dd-2dbc-48ed-b04c-6a7c8f07841e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:12.801644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a9d7dd-2dbc-48ed-b04c-6a7c8f07841e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:12.803882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 15/30] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:12.804804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9d7dd-2dbc-48ed-b04c-6a7c8f07841e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:12.815587 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:12.815857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584aec7-5bf4-48d5-8e36-78efe8ab2c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:12.816679 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584aec7-5bf4-48d5-8e36-78efe8ab2c4e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:12.817100 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584aec7-5bf4-48d5-8e36-78efe8ab2c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:12.818293 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584aec7-5bf4-48d5-8e36-78efe8ab2c4e None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:12.819219 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584aec7-5bf4-48d5-8e36-78efe8ab2c4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:12.819839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584aec7-5bf4-48d5-8e36-78efe8ab2c4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:12.821433 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584aec7-5bf4-48d5-8e36-78efe8ab2c4e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:12.839071 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 16/31] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:12.842106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfa9aa-bde3-4249-b50b-0f0af181bb91 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:12.842581 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfa9aa-bde3-4249-b50b-0f0af181bb91 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:12.843050 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdfa9aa-bde3-4249-b50b-0f0af181bb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:12.924628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a9d7dd-2dbc-48ed-b04c-6a7c8f07841e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:12.963667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658cd88-731d-48e2-b1fd-7e1f77ff40b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:12.964326 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658cd88-731d-48e2-b1fd-7e1f77ff40b0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:12.964606 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658cd88-731d-48e2-b1fd-7e1f77ff40b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:12.964878 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658cd88-731d-48e2-b1fd-7e1f77ff40b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:12.965605 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658cd88-731d-48e2-b1fd-7e1f77ff40b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:12.966000 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658cd88-731d-48e2-b1fd-7e1f77ff40b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:12.967400 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a658cd88-731d-48e2-b1fd-7e1f77ff40b0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:12.977666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3bdfa9aa-bde3-4249-b50b-0f0af181bb91 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:12.979124 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3bdfa9aa-bde3-4249-b50b-0f0af181bb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:12.980844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 16/32] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:12.982669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658cd88-731d-48e2-b1fd-7e1f77ff40b0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:13.000530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:13.072580 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 17/33] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:13.073514 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdfa9aa-bde3-4249-b50b-0f0af181bb91 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:13.082186 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:13.127604 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658cd88-731d-48e2-b1fd-7e1f77ff40b0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:13.175435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f7b86-edf1-419c-b88c-0c87da7c70e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:13.176003 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f7b86-edf1-419c-b88c-0c87da7c70e1 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:13.176294 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f7b86-edf1-419c-b88c-0c87da7c70e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:13.176574 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4f7b86-edf1-419c-b88c-0c87da7c70e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:13.177342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f7b86-edf1-419c-b88c-0c87da7c70e1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:13.178890 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4f7b86-edf1-419c-b88c-0c87da7c70e1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:13.214838 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdfa9aa-bde3-4249-b50b-0f0af181bb91 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:13.253180 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad26eb-6dc8-4252-9831-441f37dd32e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:13.254214 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad26eb-6dc8-4252-9831-441f37dd32e1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:13.254756 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad26eb-6dc8-4252-9831-441f37dd32e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:13.255203 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad26eb-6dc8-4252-9831-441f37dd32e1 None admin] PATH_INFO: `/v3/users/0b20c2888f9642efb7858178880ed9a0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:13.256606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad26eb-6dc8-4252-9831-441f37dd32e1 None admin] RBAC: Authorizing `identity:get_user(user_id=0b20c2888f9642efb7858178880ed9a0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:13.258290 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad26eb-6dc8-4252-9831-441f37dd32e1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:13.260511 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 18/34] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:13 2022] GET /identity/v3/users/0b20c2888f9642efb7858178880ed9a0 => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:13.261578 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad26eb-6dc8-4252-9831-441f37dd32e1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:13.282858 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:13.332787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 17/35] 10.222.0.39 () {60 vars in 1131 bytes} [Thu Jul 28 22:14:12 2022] POST /identity/v3/roles => generated 219 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:14:13.334188 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f7b86-edf1-419c-b88c-0c87da7c70e1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:13.420339 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad26eb-6dc8-4252-9831-441f37dd32e1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:13.490827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0bc5b6-927d-4f4f-95a0-3bea6989a854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:13.492183 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bc5b6-927d-4f4f-95a0-3bea6989a854 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:13.493997 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bc5b6-927d-4f4f-95a0-3bea6989a854 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:13.494533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0bc5b6-927d-4f4f-95a0-3bea6989a854 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:13.495453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0bc5b6-927d-4f4f-95a0-3bea6989a854 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:13.497934 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0bc5b6-927d-4f4f-95a0-3bea6989a854 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:13.538766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4f7b86-edf1-419c-b88c-0c87da7c70e1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:13.574227 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:13.595457 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca0fb9-ce70-4f4a-a3c9-ad7cc138765e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:13.596040 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca0fb9-ce70-4f4a-a3c9-ad7cc138765e None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:13.596310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca0fb9-ce70-4f4a-a3c9-ad7cc138765e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:13.596606 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca0fb9-ce70-4f4a-a3c9-ad7cc138765e None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:13.597309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca0fb9-ce70-4f4a-a3c9-ad7cc138765e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:13.598915 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfca0fb9-ce70-4f4a-a3c9-ad7cc138765e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:13.616512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 18/36] 10.222.0.39 () {60 vars in 1131 bytes} [Thu Jul 28 22:14:13 2022] POST /identity/v3/roles => generated 225 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:14:13.617304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca0fb9-ce70-4f4a-a3c9-ad7cc138765e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:13.678873 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 19/37] 10.222.0.39 () {60 vars in 1141 bytes} [Thu Jul 28 22:14:13 2022] POST /identity/v3/projects => generated 301 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:14:13.681158 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94f132-1435-4880-953c-9946355076b3 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:13.681491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94f132-1435-4880-953c-9946355076b3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:13.682272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca94f132-1435-4880-953c-9946355076b3 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:13.684019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 20/38] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:13.690570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29047c26-329d-416f-9c7f-f087d57d1c7e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:13.690897 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29047c26-329d-416f-9c7f-f087d57d1c7e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:13.691132 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29047c26-329d-416f-9c7f-f087d57d1c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:13.712911 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-29047c26-329d-416f-9c7f-f087d57d1c7e None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:13.715204 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-29047c26-329d-416f-9c7f-f087d57d1c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:13.776994 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca0fb9-ce70-4f4a-a3c9-ad7cc138765e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:13.812109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:13.820684 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 21/39] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:14:13.822764 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff524dca-d104-4578-b64b-66bc2217a4f0 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:13.823347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff524dca-d104-4578-b64b-66bc2217a4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:13.823649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff524dca-d104-4578-b64b-66bc2217a4f0 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:13.826472 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 22/40] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:13.828366 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d013ff-dd19-4a13-aa4b-f09a4dc6dc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:13.829115 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d013ff-dd19-4a13-aa4b-f09a4dc6dc78 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:13.829464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d013ff-dd19-4a13-aa4b-f09a4dc6dc78 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:13.829774 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d013ff-dd19-4a13-aa4b-f09a4dc6dc78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:13.833837 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8e4d5-33f5-427d-89e7-b8020eac2342 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:13.834133 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8e4d5-33f5-427d-89e7-b8020eac2342 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:13.834249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d013ff-dd19-4a13-aa4b-f09a4dc6dc78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:13.834417 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a8e4d5-33f5-427d-89e7-b8020eac2342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:13.836019 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d013ff-dd19-4a13-aa4b-f09a4dc6dc78 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:13.838072 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 19/41] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:14:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:13.844243 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d013ff-dd19-4a13-aa4b-f09a4dc6dc78 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:13.863069 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e5a8e4d5-33f5-427d-89e7-b8020eac2342 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:13.864302 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e5a8e4d5-33f5-427d-89e7-b8020eac2342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:13.916912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:13.943750 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 23/42] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:13.944875 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a8e4d5-33f5-427d-89e7-b8020eac2342 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:13.968779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d013ff-dd19-4a13-aa4b-f09a4dc6dc78 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:14.001351 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf067991-c4d8-4b8c-84f7-58d496acb844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:14.002051 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf067991-c4d8-4b8c-84f7-58d496acb844 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:14.002394 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf067991-c4d8-4b8c-84f7-58d496acb844 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:14.003138 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf067991-c4d8-4b8c-84f7-58d496acb844 None admin] PATH_INFO: `/v3/domains/default/users/0b20c2888f9642efb7858178880ed9a0/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:14.004910 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf067991-c4d8-4b8c-84f7-58d496acb844 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=0b20c2888f9642efb7858178880ed9a0, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:14.006402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf067991-c4d8-4b8c-84f7-58d496acb844 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:14.031819 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 20/43] 10.222.0.39 () {58 vars in 1361 bytes} [Thu Jul 28 22:14:13 2022] PUT /identity/v3/domains/default/users/0b20c2888f9642efb7858178880ed9a0/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:14:14.031819 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf067991-c4d8-4b8c-84f7-58d496acb844 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:14.063880 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a8e4d5-33f5-427d-89e7-b8020eac2342 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:14.096360 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:14.107525 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3528e-3c77-42e6-be1e-a5fbbd7f805a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:14.108565 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3528e-3c77-42e6-be1e-a5fbbd7f805a None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:14.108856 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3528e-3c77-42e6-be1e-a5fbbd7f805a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:14.109136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3528e-3c77-42e6-be1e-a5fbbd7f805a None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:14.110027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a3528e-3c77-42e6-be1e-a5fbbd7f805a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:14.111657 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a3528e-3c77-42e6-be1e-a5fbbd7f805a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:14.129511 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 24/44] 10.222.0.39 () {60 vars in 1131 bytes} [Thu Jul 28 22:14:13 2022] POST /identity/v3/roles => generated 223 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:14:14.143470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf067991-c4d8-4b8c-84f7-58d496acb844 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:14.163097 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:14.193698 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ad98ea-7051-4185-a324-80aff4045cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:14.194370 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad98ea-7051-4185-a324-80aff4045cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:14.194654 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad98ea-7051-4185-a324-80aff4045cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:14.194929 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ad98ea-7051-4185-a324-80aff4045cb7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:14.196130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad98ea-7051-4185-a324-80aff4045cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:14.197723 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ad98ea-7051-4185-a324-80aff4045cb7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:14.199819 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 21/45] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:14:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:14.207239 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3528e-3c77-42e6-be1e-a5fbbd7f805a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:14.318487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3528e-3c77-42e6-be1e-a5fbbd7f805a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:14.354175 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6895a518-1765-4d80-aa52-ace4166cd8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:14.354806 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895a518-1765-4d80-aa52-ace4166cd8ec None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:14.355092 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895a518-1765-4d80-aa52-ace4166cd8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:14.355394 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6895a518-1765-4d80-aa52-ace4166cd8ec None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:14.356095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895a518-1765-4d80-aa52-ace4166cd8ec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:14.358340 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6895a518-1765-4d80-aa52-ace4166cd8ec None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:14.383961 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 25/46] 10.222.0.39 () {60 vars in 1141 bytes} [Thu Jul 28 22:14:14 2022] POST /identity/v3/projects => generated 312 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:14:16.085677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82467272-9263-4a7a-ad2c-409a63d136af None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:16.086208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82467272-9263-4a7a-ad2c-409a63d136af None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:16.086289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82467272-9263-4a7a-ad2c-409a63d136af None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:16.087660 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 22/47] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:16.094090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f088e7-01cb-45fa-81a3-644421ce53aa None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:16.094493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f088e7-01cb-45fa-81a3-644421ce53aa None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:16.094664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f088e7-01cb-45fa-81a3-644421ce53aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:16.113095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-98f088e7-01cb-45fa-81a3-644421ce53aa None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:16.114332 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-98f088e7-01cb-45fa-81a3-644421ce53aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:16.152056 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:16.186921 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 26/48] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:16.209705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82467272-9263-4a7a-ad2c-409a63d136af None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:16.309203 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82467272-9263-4a7a-ad2c-409a63d136af None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:16.342358 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939be373-fb7d-4f68-92da-2d18a1341328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:16.342920 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939be373-fb7d-4f68-92da-2d18a1341328 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:16.343202 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939be373-fb7d-4f68-92da-2d18a1341328 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:16.343475 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939be373-fb7d-4f68-92da-2d18a1341328 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:16.353741 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939be373-fb7d-4f68-92da-2d18a1341328 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:16.358742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-939be373-fb7d-4f68-92da-2d18a1341328 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:14:16.358742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:16.358742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:16.358742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:16.358742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:16.358742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:16.358742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:16.358742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:16.358742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:16.358742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:16.358742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:16.358742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:16.358742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:16.358742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:16.358742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:16.358742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:16.359744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:16.359744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:16.359744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:16.359744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:16.359744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:16.359744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:16.359744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:16.359744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:16.359744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:16.359744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:16.359744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:16.359744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:16.359744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:16.359744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:16.359744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:16.359744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:16.360531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:16.360531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:16.360531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:16.360531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:16.360531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:16.360531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:16.360531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:16.360531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:16.360531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:16.360531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:16.360531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:14:16.360531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:16.360531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 23/49] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:14:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:16.365768 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f088e7-01cb-45fa-81a3-644421ce53aa None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:16.387416 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae759a4-b51c-4216-8006-984ea344e56e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:16.388613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae759a4-b51c-4216-8006-984ea344e56e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:16.389702 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae759a4-b51c-4216-8006-984ea344e56e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:16.391350 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 24/50] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:14:16.396797 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58180db-89cf-4f9e-889e-cd9e2762ba96 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:16.397085 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58180db-89cf-4f9e-889e-cd9e2762ba96 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:16.397362 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58180db-89cf-4f9e-889e-cd9e2762ba96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:16.416398 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f58180db-89cf-4f9e-889e-cd9e2762ba96 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:16.417970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f58180db-89cf-4f9e-889e-cd9e2762ba96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:16.454444 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:16.470068 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f088e7-01cb-45fa-81a3-644421ce53aa None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:16.487069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 25/51] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:14:16.503880 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fe1f9-cddd-44cd-a578-f2b22159f8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:16.504510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fe1f9-cddd-44cd-a578-f2b22159f8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:16.504801 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fe1f9-cddd-44cd-a578-f2b22159f8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:16.505083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7fe1f9-cddd-44cd-a578-f2b22159f8a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:16.505843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7fe1f9-cddd-44cd-a578-f2b22159f8a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:16.506158 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7fe1f9-cddd-44cd-a578-f2b22159f8a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:16.507562 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7fe1f9-cddd-44cd-a578-f2b22159f8a5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:16.510430 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58180db-89cf-4f9e-889e-cd9e2762ba96 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:16.521072 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 27/52] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:16.522317 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fe1f9-cddd-44cd-a578-f2b22159f8a5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:16.552847 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:16.609773 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58180db-89cf-4f9e-889e-cd9e2762ba96 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:16.620719 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7fe1f9-cddd-44cd-a578-f2b22159f8a5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:16.643511 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23928757-4a93-41f3-b90d-73a452932c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:16.644159 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23928757-4a93-41f3-b90d-73a452932c93 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:16.644448 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23928757-4a93-41f3-b90d-73a452932c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:16.644723 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23928757-4a93-41f3-b90d-73a452932c93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:16.645905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23928757-4a93-41f3-b90d-73a452932c93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:16.647360 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23928757-4a93-41f3-b90d-73a452932c93 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:16.649108 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 26/53] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:14:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:16.654483 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23928757-4a93-41f3-b90d-73a452932c93 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:16.656191 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3025a-954a-491f-b2a2-ceaf814f19a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:16.656842 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3025a-954a-491f-b2a2-ceaf814f19a1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:16.657133 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3025a-954a-491f-b2a2-ceaf814f19a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:16.657408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3025a-954a-491f-b2a2-ceaf814f19a1 None admin] PATH_INFO: `/v3/users/0b20c2888f9642efb7858178880ed9a0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:16.658643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3025a-954a-491f-b2a2-ceaf814f19a1 None admin] RBAC: Authorizing `identity:get_user(user_id=0b20c2888f9642efb7858178880ed9a0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:16.660099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3025a-954a-491f-b2a2-ceaf814f19a1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:16.661743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 28/54] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:16 2022] GET /identity/v3/users/0b20c2888f9642efb7858178880ed9a0 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:16.666942 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3025a-954a-491f-b2a2-ceaf814f19a1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:16.694824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:16.753862 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23928757-4a93-41f3-b90d-73a452932c93 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:16.768610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:16.772325 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3025a-954a-491f-b2a2-ceaf814f19a1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:16.795705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f3a23-743f-438b-95c1-55b3147c2e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:16.796338 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f3a23-743f-438b-95c1-55b3147c2e52 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:16.796626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f3a23-743f-438b-95c1-55b3147c2e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:16.796908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f3a23-743f-438b-95c1-55b3147c2e52 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:16.797612 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f3a23-743f-438b-95c1-55b3147c2e52 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:16.799063 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f3a23-743f-438b-95c1-55b3147c2e52 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:16.811594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c65b9-dc91-4c06-825b-ac54762b9d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:16.812246 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c65b9-dc91-4c06-825b-ac54762b9d74 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:16.812539 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c65b9-dc91-4c06-825b-ac54762b9d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:16.812820 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18c65b9-dc91-4c06-825b-ac54762b9d74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:16.814034 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c65b9-dc91-4c06-825b-ac54762b9d74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:16.815495 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18c65b9-dc91-4c06-825b-ac54762b9d74 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:16.817360 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 29/55] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:14:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:16.822806 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c65b9-dc91-4c06-825b-ac54762b9d74 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:16.827993 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 27/56] 10.222.0.39 () {60 vars in 1140 bytes} [Thu Jul 28 22:14:16 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:14:16.933890 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18c65b9-dc91-4c06-825b-ac54762b9d74 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:16.971409 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f11e81c-123d-4a3b-a013-bff82dc4d5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:16.972042 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11e81c-123d-4a3b-a013-bff82dc4d5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:16.972335 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11e81c-123d-4a3b-a013-bff82dc4d5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:16.972614 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f11e81c-123d-4a3b-a013-bff82dc4d5b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:16.973417 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11e81c-123d-4a3b-a013-bff82dc4d5b5 None admin] RBAC: Adding query filter params (role.id=aded7bd43e974686b3c23bd9d4255b24, scope.domain.id=default, user.id=0b20c2888f9642efb7858178880ed9a0) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:16.973764 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11e81c-123d-4a3b-a013-bff82dc4d5b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:16.975229 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f11e81c-123d-4a3b-a013-bff82dc4d5b5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:16.981172 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 30/57] 10.222.0.39 () {56 vars in 1322 bytes} [Thu Jul 28 22:14:16 2022] GET /identity/v3/role_assignments?user.id=0b20c2888f9642efb7858178880ed9a0&scope.domain.id=default&role.id=aded7bd43e974686b3c23bd9d4255b24 => generated 530 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:18.835358 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9b6db-e59e-4a76-95be-1a8fad1ff6a7 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:18.835900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9b6db-e59e-4a76-95be-1a8fad1ff6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:18.835900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9b6db-e59e-4a76-95be-1a8fad1ff6a7 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:18.837275 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 28/58] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:18.842821 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a92897-e7f8-43b6-82eb-260ec239ff5f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:18.843226 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a92897-e7f8-43b6-82eb-260ec239ff5f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:18.843392 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a92897-e7f8-43b6-82eb-260ec239ff5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:18.861806 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d8a92897-e7f8-43b6-82eb-260ec239ff5f None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:18.863006 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d8a92897-e7f8-43b6-82eb-260ec239ff5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:18.887762 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:18.927862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 31/59] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:18.952618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9b6db-e59e-4a76-95be-1a8fad1ff6a7 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:19.043389 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:19.053880 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9b6db-e59e-4a76-95be-1a8fad1ff6a7 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:19.086665 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1765ff-e02c-4e3f-a085-31bc9bf42506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:19.087246 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1765ff-e02c-4e3f-a085-31bc9bf42506 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:19.087532 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1765ff-e02c-4e3f-a085-31bc9bf42506 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:19.087810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1765ff-e02c-4e3f-a085-31bc9bf42506 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:19.088960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1765ff-e02c-4e3f-a085-31bc9bf42506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:19.090449 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1765ff-e02c-4e3f-a085-31bc9bf42506 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:19.092584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 29/60] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:14:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:19.098006 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a92897-e7f8-43b6-82eb-260ec239ff5f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:19.192903 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a92897-e7f8-43b6-82eb-260ec239ff5f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:19.225789 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc9e821-5979-42d0-be60-c2a8449cd4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:19.226404 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9e821-5979-42d0-be60-c2a8449cd4f6 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:19.226686 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9e821-5979-42d0-be60-c2a8449cd4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:19.226961 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc9e821-5979-42d0-be60-c2a8449cd4f6 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:19.227649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc9e821-5979-42d0-be60-c2a8449cd4f6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:19.229087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc9e821-5979-42d0-be60-c2a8449cd4f6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:19.254826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 32/61] 10.222.0.39 () {60 vars in 1132 bytes} [Thu Jul 28 22:14:19 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:14:22.370871 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93560315-2b77-47fe-aeb2-282538cc03e7 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:22.371414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93560315-2b77-47fe-aeb2-282538cc03e7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:22.371414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93560315-2b77-47fe-aeb2-282538cc03e7 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:22.372994 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 30/62] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:22.378221 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2eafee-cc6e-483c-b46d-497b21155151 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:22.378650 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2eafee-cc6e-483c-b46d-497b21155151 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:22.378817 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2eafee-cc6e-483c-b46d-497b21155151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:22.410353 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a6609-eec8-4c66-9aea-326ed59aa23a None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:22.410626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a6609-eec8-4c66-9aea-326ed59aa23a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:22.410839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7a6609-eec8-4c66-9aea-326ed59aa23a None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:22.412468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 31/63] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:22.413796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9c2eafee-cc6e-483c-b46d-497b21155151 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:22.415009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9c2eafee-cc6e-483c-b46d-497b21155151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:22.417126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c304b-01ff-42af-acc5-f0976134134a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:22.417412 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c304b-01ff-42af-acc5-f0976134134a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:22.417727 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97c304b-01ff-42af-acc5-f0976134134a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:22.445346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d97c304b-01ff-42af-acc5-f0976134134a None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:22.446582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d97c304b-01ff-42af-acc5-f0976134134a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:22.453214 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:22.558892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 33/64] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:22.568269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0739d4a-533e-410d-bde6-5f95fb8b0c25 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:22.569433 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0739d4a-533e-410d-bde6-5f95fb8b0c25 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:22.570106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0739d4a-533e-410d-bde6-5f95fb8b0c25 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:22.572975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 34/65] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:22.578868 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 32/66] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:14:22.580938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bb25f-0c10-4684-8b7f-52e79e010b1c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:22.581452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bb25f-0c10-4684-8b7f-52e79e010b1c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:22.581988 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7bb25f-0c10-4684-8b7f-52e79e010b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:22.585856 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c304b-01ff-42af-acc5-f0976134134a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:22.605030 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-dd7bb25f-0c10-4684-8b7f-52e79e010b1c None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:22.607323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-dd7bb25f-0c10-4684-8b7f-52e79e010b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:22.692691 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 35/67] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:14:22.693499 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bb25f-0c10-4684-8b7f-52e79e010b1c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:22.708310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:22.726021 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97c304b-01ff-42af-acc5-f0976134134a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:22.771640 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92218d-8b2b-4045-baa1-83b138965713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:22.773498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92218d-8b2b-4045-baa1-83b138965713 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:22.773908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92218d-8b2b-4045-baa1-83b138965713 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:22.774284 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a92218d-8b2b-4045-baa1-83b138965713 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:22.791950 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a92218d-8b2b-4045-baa1-83b138965713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:22.801286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-6a92218d-8b2b-4045-baa1-83b138965713 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:14:22.801286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:22.801286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:22.801286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:22.801286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:22.801286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:22.801286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:22.801286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:22.801286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:22.801286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:22.801286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:22.801286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:22.801286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:22.801286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:22.801286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:22.801286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:22.802224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:22.802224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:22.802224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:22.802224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:22.802224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:22.802224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:22.802224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:22.802224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:22.802224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:22.802224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:22.802224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:22.802224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:22.802224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:22.802224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:22.802224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:22.802224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:22.803012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:22.803012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:22.803012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:22.803012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:22.803012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:22.803012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:22.803012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:22.803012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:22.803012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:22.803012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:22.803012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:14:22.803012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:22.803012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 33/68] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:14:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:22.803643 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92218d-8b2b-4045-baa1-83b138965713 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:22.817131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:22.844611 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7bb25f-0c10-4684-8b7f-52e79e010b1c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:22.882924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab241b-d91a-43fe-9768-5d70efee2e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:22.883926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab241b-d91a-43fe-9768-5d70efee2e19 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:22.884379 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab241b-d91a-43fe-9768-5d70efee2e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:22.884911 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ab241b-d91a-43fe-9768-5d70efee2e19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:22.896741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ab241b-d91a-43fe-9768-5d70efee2e19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:22.902197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-01ab241b-d91a-43fe-9768-5d70efee2e19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:22.902197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:22.902197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:22.902197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:22.902197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:22.902197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:22.902197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:22.902197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:22.902197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:22.902197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:22.902197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:22.902197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:22.902197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:22.902197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:22.902197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:22.902197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:22.903804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:22.903804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:22.903804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:22.903804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:22.903804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:22.903804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:22.903804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:22.903804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:22.903804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:22.903804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:22.903804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:22.903804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:22.903804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:22.903804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:22.903804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:22.903804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:22.904570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:22.904570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:22.904570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:22.904570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:22.904570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:22.904570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:22.904570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:22.904570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:22.904570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:22.904570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:22.904570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:22.904570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:14:22.906975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 36/69] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:22.908011 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab241b-d91a-43fe-9768-5d70efee2e19 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:22.936387 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a92218d-8b2b-4045-baa1-83b138965713 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:22.974133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5bf7c-0b79-4a48-a6e3-7888c52ee3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:22.975067 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5bf7c-0b79-4a48-a6e3-7888c52ee3aa None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:22.975534 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5bf7c-0b79-4a48-a6e3-7888c52ee3aa None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:22.976021 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e5bf7c-0b79-4a48-a6e3-7888c52ee3aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:22.987214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e5bf7c-0b79-4a48-a6e3-7888c52ee3aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:22.992473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-44e5bf7c-0b79-4a48-a6e3-7888c52ee3aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:14:22.992473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:22.992473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:22.992473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:22.992473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:22.992473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:22.992473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:22.992473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:22.992473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:22.992473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:22.992473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:22.992473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:22.992473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:22.992473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:22.992473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:22.992473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:22.993392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:22.993392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:22.993392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:22.993392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:22.993392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:22.993392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:22.993392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:22.993392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:22.993392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:22.993392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:22.993392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:22.993392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:22.993392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:22.993392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:22.993392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:22.993392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:22.994191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:22.994191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:22.994191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:22.994191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:22.994191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:22.994191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:22.994191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:22.994191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:22.994191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:22.994191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:22.994191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:14:22.994191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:22.995619 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 34/70] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:14:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:22.996362 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5bf7c-0b79-4a48-a6e3-7888c52ee3aa None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:23.011558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:23.034407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ab241b-d91a-43fe-9768-5d70efee2e19 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:23.070683 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee42ea1-c84a-42a3-a958-77ff0e5591bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:23.071314 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee42ea1-c84a-42a3-a958-77ff0e5591bf None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:23.071603 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee42ea1-c84a-42a3-a958-77ff0e5591bf None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:23.071886 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee42ea1-c84a-42a3-a958-77ff0e5591bf None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:23.072615 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee42ea1-c84a-42a3-a958-77ff0e5591bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:23.072932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee42ea1-c84a-42a3-a958-77ff0e5591bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:23.074386 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee42ea1-c84a-42a3-a958-77ff0e5591bf None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:23.080100 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:23.085892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 37/71] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:23.087237 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277a770-b3de-4fe0-9112-de3793bbe070 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:23.087511 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277a770-b3de-4fe0-9112-de3793bbe070 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:23.088269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f277a770-b3de-4fe0-9112-de3793bbe070 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:23.089568 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 38/72] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:23.090272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277a770-b3de-4fe0-9112-de3793bbe070 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:23.109692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e5bf7c-0b79-4a48-a6e3-7888c52ee3aa None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:23.145040 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991872d-2385-4f1d-a7b5-afc551c34087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:23.145708 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991872d-2385-4f1d-a7b5-afc551c34087 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:23.145999 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991872d-2385-4f1d-a7b5-afc551c34087 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:23.146281 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991872d-2385-4f1d-a7b5-afc551c34087 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:23.147013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991872d-2385-4f1d-a7b5-afc551c34087 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:23.147331 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991872d-2385-4f1d-a7b5-afc551c34087 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:23.148734 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991872d-2385-4f1d-a7b5-afc551c34087 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:23.160558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 35/73] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:23.161876 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d228a7e-7d44-4ec1-b870-a2b6373ef8b1 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:23.162177 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d228a7e-7d44-4ec1-b870-a2b6373ef8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:23.162461 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d228a7e-7d44-4ec1-b870-a2b6373ef8b1 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:23.163686 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 36/74] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:23.164453 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d228a7e-7d44-4ec1-b870-a2b6373ef8b1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:23.211142 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f277a770-b3de-4fe0-9112-de3793bbe070 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:23.236492 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:23.245399 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5979b3-80cc-4063-9f67-3545b82b4132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:23.246057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5979b3-80cc-4063-9f67-3545b82b4132 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:23.246348 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5979b3-80cc-4063-9f67-3545b82b4132 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:23.247227 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5979b3-80cc-4063-9f67-3545b82b4132 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:23.248083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5979b3-80cc-4063-9f67-3545b82b4132 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:23.248392 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5979b3-80cc-4063-9f67-3545b82b4132 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:23.249805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5979b3-80cc-4063-9f67-3545b82b4132 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:23.257855 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 39/75] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:14:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:23.257855 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ec469-81f7-4431-9af3-427ba80854ff None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:23.257855 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ec469-81f7-4431-9af3-427ba80854ff None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:23.257855 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ec469-81f7-4431-9af3-427ba80854ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:23.269472 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d228a7e-7d44-4ec1-b870-a2b6373ef8b1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:23.277212 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-469ec469-81f7-4431-9af3-427ba80854ff None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:23.278966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-469ec469-81f7-4431-9af3-427ba80854ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:23.303700 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f39d83-7e39-4683-a50e-332986b53b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:23.304425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f39d83-7e39-4683-a50e-332986b53b1e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:23.304779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f39d83-7e39-4683-a50e-332986b53b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:23.305068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f39d83-7e39-4683-a50e-332986b53b1e None admin] PATH_INFO: `/v3/users/0b20c2888f9642efb7858178880ed9a0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:23.306285 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f39d83-7e39-4683-a50e-332986b53b1e None admin] RBAC: Authorizing `identity:get_user(user_id=0b20c2888f9642efb7858178880ed9a0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:23.307747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f39d83-7e39-4683-a50e-332986b53b1e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:23.309391 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 37/76] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:23 2022] GET /identity/v3/users/0b20c2888f9642efb7858178880ed9a0 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:23.310238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f39d83-7e39-4683-a50e-332986b53b1e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:23.334512 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:23.346217 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 40/77] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:23.347443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c622c-fad3-4da2-99f4-6f0c73e9fc10 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:23.347732 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c622c-fad3-4da2-99f4-6f0c73e9fc10 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:23.348009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c622c-fad3-4da2-99f4-6f0c73e9fc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:23.365694 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-4b9c622c-fad3-4da2-99f4-6f0c73e9fc10 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:23.366872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4b9c622c-fad3-4da2-99f4-6f0c73e9fc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:23.428432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f39d83-7e39-4683-a50e-332986b53b1e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:23.435107 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 41/78] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:23.436040 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c622c-fad3-4da2-99f4-6f0c73e9fc10 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:23.464293 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213a67d-8b67-45b7-a674-267e542e980c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:23.464937 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213a67d-8b67-45b7-a674-267e542e980c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:23.465220 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213a67d-8b67-45b7-a674-267e542e980c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:23.466587 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213a67d-8b67-45b7-a674-267e542e980c None admin] PATH_INFO: `/v3/users/f7fc4b88abc344d2b72a4653dc811819` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:23.476736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:23.481721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213a67d-8b67-45b7-a674-267e542e980c None admin] RBAC: Authorizing `identity:get_user(user_id=f7fc4b88abc344d2b72a4653dc811819)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:23.483193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213a67d-8b67-45b7-a674-267e542e980c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:23.484849 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 38/79] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:23 2022] GET /identity/v3/users/f7fc4b88abc344d2b72a4653dc811819 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:23.485626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213a67d-8b67-45b7-a674-267e542e980c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:23.541187 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c622c-fad3-4da2-99f4-6f0c73e9fc10 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:23.563283 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:23.574602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bb17e-aa8b-46dd-9e9c-4c265f31da1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:23.575914 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bb17e-aa8b-46dd-9e9c-4c265f31da1e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:23.576200 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bb17e-aa8b-46dd-9e9c-4c265f31da1e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:23.576477 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bb17e-aa8b-46dd-9e9c-4c265f31da1e None admin] PATH_INFO: `/v3/users/f7fc4b88abc344d2b72a4653dc811819` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:23.577686 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923bb17e-aa8b-46dd-9e9c-4c265f31da1e None admin] RBAC: Authorizing `identity:get_user(user_id=f7fc4b88abc344d2b72a4653dc811819)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:23.579135 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923bb17e-aa8b-46dd-9e9c-4c265f31da1e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:23.580751 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 42/80] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:23 2022] GET /identity/v3/users/f7fc4b88abc344d2b72a4653dc811819 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:23.582414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bb17e-aa8b-46dd-9e9c-4c265f31da1e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:23.583030 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213a67d-8b67-45b7-a674-267e542e980c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:23.617153 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34cf4e-5edb-4fd3-8d4d-229d8513b703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:23.617812 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34cf4e-5edb-4fd3-8d4d-229d8513b703 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:23.618101 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34cf4e-5edb-4fd3-8d4d-229d8513b703 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:23.618377 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34cf4e-5edb-4fd3-8d4d-229d8513b703 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:23.683515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:23.687429 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34cf4e-5edb-4fd3-8d4d-229d8513b703 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:23.688490 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bb17e-aa8b-46dd-9e9c-4c265f31da1e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:23.689381 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34cf4e-5edb-4fd3-8d4d-229d8513b703 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:23.722026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc232f76-f5c3-42d7-bc5c-6a8e981d66d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:23.722665 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc232f76-f5c3-42d7-bc5c-6a8e981d66d9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:23.722953 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc232f76-f5c3-42d7-bc5c-6a8e981d66d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:23.723232 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc232f76-f5c3-42d7-bc5c-6a8e981d66d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:23.733078 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc232f76-f5c3-42d7-bc5c-6a8e981d66d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:23.738553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-fc232f76-f5c3-42d7-bc5c-6a8e981d66d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:23.738553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:23.738553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:23.738553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:23.738553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:23.738553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:23.738553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:23.738553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:23.738553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:23.738553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:23.738553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:23.738553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:23.738553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:23.738553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:23.738553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:23.738553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:23.739634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:23.739634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:23.739634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:23.739634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:23.739634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:23.739634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:23.739634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:23.739634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:23.739634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:23.739634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:23.739634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:23.739634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:23.739634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:23.739634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:23.739634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:23.739634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:23.740393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:23.740393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:23.740393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:23.740393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:23.740393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:23.740393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:23.740393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:23.740393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:23.740393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:23.740393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:23.740393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:23.740393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:14:23.740393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 43/81] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:23.741010 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc232f76-f5c3-42d7-bc5c-6a8e981d66d9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:23.789194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 39/82] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:14:23.789999 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34cf4e-5edb-4fd3-8d4d-229d8513b703 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:23.797224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:23.846477 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc232f76-f5c3-42d7-bc5c-6a8e981d66d9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:23.879219 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980238f-c6a1-480f-a885-6c9987294aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:23.879863 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980238f-c6a1-480f-a885-6c9987294aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:23.880150 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980238f-c6a1-480f-a885-6c9987294aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:23.880437 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4980238f-c6a1-480f-a885-6c9987294aa2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:23.881621 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980238f-c6a1-480f-a885-6c9987294aa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:23.883088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4980238f-c6a1-480f-a885-6c9987294aa2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:23.884869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 44/83] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:14:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:23.885409 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:23.885546 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980238f-c6a1-480f-a885-6c9987294aa2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:23.894024 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34cf4e-5edb-4fd3-8d4d-229d8513b703 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:23.928334 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d21738-f97e-4ab5-83e7-0d8bfb3f7932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:23.928957 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d21738-f97e-4ab5-83e7-0d8bfb3f7932 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:23.929246 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d21738-f97e-4ab5-83e7-0d8bfb3f7932 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:23.929553 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d21738-f97e-4ab5-83e7-0d8bfb3f7932 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:23.982980 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4980238f-c6a1-480f-a885-6c9987294aa2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:23.998171 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d21738-f97e-4ab5-83e7-0d8bfb3f7932 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:23.999675 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d21738-f97e-4ab5-83e7-0d8bfb3f7932 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:24.000312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:24.018728 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41743919-48be-48fd-bccd-9372c6b2e168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:24.019360 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41743919-48be-48fd-bccd-9372c6b2e168 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:24.019655 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41743919-48be-48fd-bccd-9372c6b2e168 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:24.019929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41743919-48be-48fd-bccd-9372c6b2e168 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:24.087130 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41743919-48be-48fd-bccd-9372c6b2e168 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:24.088572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41743919-48be-48fd-bccd-9372c6b2e168 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:24.098149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:24.105471 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 40/84] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:14:24.106798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d21738-f97e-4ab5-83e7-0d8bfb3f7932 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:24.191254 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 45/85] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:14:24.196456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41743919-48be-48fd-bccd-9372c6b2e168 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:24.209820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d21738-f97e-4ab5-83e7-0d8bfb3f7932 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:24.215291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:24.244035 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b548f-cd05-4148-a391-83dd52744ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:24.244665 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b548f-cd05-4148-a391-83dd52744ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:24.244952 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b548f-cd05-4148-a391-83dd52744ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:24.245231 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b548f-cd05-4148-a391-83dd52744ff1 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:24.245990 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b548f-cd05-4148-a391-83dd52744ff1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:24.246304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b548f-cd05-4148-a391-83dd52744ff1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:24.248538 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b548f-cd05-4148-a391-83dd52744ff1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:24.261026 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 41/86] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:24.261026 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b548f-cd05-4148-a391-83dd52744ff1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:24.302963 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41743919-48be-48fd-bccd-9372c6b2e168 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:24.314697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:24.336813 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cec69-d6ef-4fc9-9970-a876aa06eb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:24.337599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cec69-d6ef-4fc9-9970-a876aa06eb07 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:24.337857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cec69-d6ef-4fc9-9970-a876aa06eb07 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:24.338139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cec69-d6ef-4fc9-9970-a876aa06eb07 None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:24.339323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22cec69-d6ef-4fc9-9970-a876aa06eb07 None admin] RBAC: Authorizing `identity:get_project(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:24.340898 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22cec69-d6ef-4fc9-9970-a876aa06eb07 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:24.342611 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 46/87] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:24 2022] GET /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:24.343376 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cec69-d6ef-4fc9-9970-a876aa06eb07 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:24.359654 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b548f-cd05-4148-a391-83dd52744ff1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:24.393640 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac85b90-0773-41f8-92e4-41764555b7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:24.394259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac85b90-0773-41f8-92e4-41764555b7a9 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:24.394542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac85b90-0773-41f8-92e4-41764555b7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:24.394816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac85b90-0773-41f8-92e4-41764555b7a9 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:24.395490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac85b90-0773-41f8-92e4-41764555b7a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:24.396929 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac85b90-0773-41f8-92e4-41764555b7a9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:24.426658 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 42/88] 10.222.0.39 () {60 vars in 1132 bytes} [Thu Jul 28 22:14:24 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:14:24.427502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac85b90-0773-41f8-92e4-41764555b7a9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:24.429625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:24.451598 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cec69-d6ef-4fc9-9970-a876aa06eb07 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:24.492028 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ab569-c84c-4340-80d8-fa817f3b1497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:24.492703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ab569-c84c-4340-80d8-fa817f3b1497 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:24.492994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ab569-c84c-4340-80d8-fa817f3b1497 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:24.493277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317ab569-c84c-4340-80d8-fa817f3b1497 None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:24.494827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ab569-c84c-4340-80d8-fa817f3b1497 None admin] RBAC: Authorizing `identity:get_project(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:24.496339 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317ab569-c84c-4340-80d8-fa817f3b1497 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:24.504018 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 47/89] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:24 2022] GET /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:24.504018 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ab569-c84c-4340-80d8-fa817f3b1497 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:24.548769 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac85b90-0773-41f8-92e4-41764555b7a9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:24.568540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:24.596456 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c400e0-0145-496e-9b6c-5bf0c6b4c4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:24.597507 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c400e0-0145-496e-9b6c-5bf0c6b4c4cb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:24.597507 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c400e0-0145-496e-9b6c-5bf0c6b4c4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:24.597507 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c400e0-0145-496e-9b6c-5bf0c6b4c4cb None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:24.599441 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c400e0-0145-496e-9b6c-5bf0c6b4c4cb None admin] RBAC: Authorizing `identity:get_project(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:24.600940 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c400e0-0145-496e-9b6c-5bf0c6b4c4cb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:24.602830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 43/90] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:24 2022] GET /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:24.608445 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c400e0-0145-496e-9b6c-5bf0c6b4c4cb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:24.622198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317ab569-c84c-4340-80d8-fa817f3b1497 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:24.666839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b817f8-5883-449c-b277-4718ca8858a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:24.667610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b817f8-5883-449c-b277-4718ca8858a5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:24.667964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b817f8-5883-449c-b277-4718ca8858a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:24.669471 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b817f8-5883-449c-b277-4718ca8858a5 None admin] PATH_INFO: `/v3/users/f7fc4b88abc344d2b72a4653dc811819` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:24.670350 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b817f8-5883-449c-b277-4718ca8858a5 None admin] RBAC: Authorizing `identity:get_user(user_id=f7fc4b88abc344d2b72a4653dc811819)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:24.671933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b817f8-5883-449c-b277-4718ca8858a5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:24.673742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 48/91] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:24 2022] GET /identity/v3/users/f7fc4b88abc344d2b72a4653dc811819 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:24.674605 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b817f8-5883-449c-b277-4718ca8858a5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:24.706889 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:24.728887 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c400e0-0145-496e-9b6c-5bf0c6b4c4cb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:24.775645 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782013b-8749-46f2-86cc-b3bb46fe9330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:24.776269 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782013b-8749-46f2-86cc-b3bb46fe9330 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:24.776556 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782013b-8749-46f2-86cc-b3bb46fe9330 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:24.776828 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6782013b-8749-46f2-86cc-b3bb46fe9330 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:24.777673 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782013b-8749-46f2-86cc-b3bb46fe9330 None admin] RBAC: Adding query filter params (role.id=aded7bd43e974686b3c23bd9d4255b24, scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29, user.id=0b20c2888f9642efb7858178880ed9a0) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:24.778572 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782013b-8749-46f2-86cc-b3bb46fe9330 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:24.780040 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6782013b-8749-46f2-86cc-b3bb46fe9330 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:24.787473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 44/92] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:14:24 2022] GET /identity/v3/role_assignments?user.id=0b20c2888f9642efb7858178880ed9a0&scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29&role.id=aded7bd43e974686b3c23bd9d4255b24 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:24.788295 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782013b-8749-46f2-86cc-b3bb46fe9330 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:24.813404 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b817f8-5883-449c-b277-4718ca8858a5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:24.844253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:24.855400 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7e970c-2547-48ba-a7e2-d6bcb69bab8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:24.857052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e970c-2547-48ba-a7e2-d6bcb69bab8f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:24.857962 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e970c-2547-48ba-a7e2-d6bcb69bab8f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:24.858709 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7e970c-2547-48ba-a7e2-d6bcb69bab8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:24.859933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7e970c-2547-48ba-a7e2-d6bcb69bab8f None admin] RBAC: Adding query filter params (role.id=4912a35790fe4d4297540156083d81d1, scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29, user.id=f7fc4b88abc344d2b72a4653dc811819) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:24.860483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7e970c-2547-48ba-a7e2-d6bcb69bab8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:24.862746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7e970c-2547-48ba-a7e2-d6bcb69bab8f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:24.871236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 49/93] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:14:24 2022] GET /identity/v3/role_assignments?user.id=f7fc4b88abc344d2b72a4653dc811819&scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29&role.id=4912a35790fe4d4297540156083d81d1 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:24.873387 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7e970c-2547-48ba-a7e2-d6bcb69bab8f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:24.913413 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6782013b-8749-46f2-86cc-b3bb46fe9330 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:24.970416 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ebce4-51fd-4f85-b281-0fa89a2e2c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:24.971080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ebce4-51fd-4f85-b281-0fa89a2e2c1e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:24.971371 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ebce4-51fd-4f85-b281-0fa89a2e2c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:24.971656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ebce4-51fd-4f85-b281-0fa89a2e2c1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:24.972491 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ebce4-51fd-4f85-b281-0fa89a2e2c1e None admin] RBAC: Adding query filter params (role.id=4146966c23674402aebb9a5b21cf18ba, scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29, user.id=f7fc4b88abc344d2b72a4653dc811819) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:24.972810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ebce4-51fd-4f85-b281-0fa89a2e2c1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:24.974382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ebce4-51fd-4f85-b281-0fa89a2e2c1e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:24.984588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:24.988298 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 45/94] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:14:24 2022] GET /identity/v3/role_assignments?user.id=f7fc4b88abc344d2b72a4653dc811819&scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29&role.id=4146966c23674402aebb9a5b21cf18ba => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:25.004656 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7e970c-2547-48ba-a7e2-d6bcb69bab8f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:25.048785 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3918c-1550-4e05-977c-bc18cfb84376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:25.049554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3918c-1550-4e05-977c-bc18cfb84376 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:25.049858 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3918c-1550-4e05-977c-bc18cfb84376 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:25.050135 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3918c-1550-4e05-977c-bc18cfb84376 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:25.132416 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3918c-1550-4e05-977c-bc18cfb84376 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:25.133880 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d3918c-1550-4e05-977c-bc18cfb84376 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:25.232681 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:25.251045 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 50/95] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:14:25.256750 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ebce4-51fd-4f85-b281-0fa89a2e2c1e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:25.379837 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ebce4-51fd-4f85-b281-0fa89a2e2c1e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:25.393678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:25.422767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98663dd9-0ab5-4748-b04d-c2fc67945308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:25.423480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98663dd9-0ab5-4748-b04d-c2fc67945308 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:25.423767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98663dd9-0ab5-4748-b04d-c2fc67945308 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:25.424100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98663dd9-0ab5-4748-b04d-c2fc67945308 None admin] PATH_INFO: `/v3/projects/cee19bd514184a9988cbe093840d4283` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:25.425300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98663dd9-0ab5-4748-b04d-c2fc67945308 None admin] RBAC: Authorizing `identity:get_project(project_id=cee19bd514184a9988cbe093840d4283)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:25.426786 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98663dd9-0ab5-4748-b04d-c2fc67945308 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:25.428463 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 46/96] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:25 2022] GET /identity/v3/projects/cee19bd514184a9988cbe093840d4283 => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:25.435009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3918c-1550-4e05-977c-bc18cfb84376 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:25.567744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d3918c-1550-4e05-977c-bc18cfb84376 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:25.613040 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dd1779-78aa-4a41-96fd-76fd1f34460d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:25.613698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd1779-78aa-4a41-96fd-76fd1f34460d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:25.613984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd1779-78aa-4a41-96fd-76fd1f34460d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:25.614256 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dd1779-78aa-4a41-96fd-76fd1f34460d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:25.615164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd1779-78aa-4a41-96fd-76fd1f34460d None admin] RBAC: Adding query filter params (role.id=4912a35790fe4d4297540156083d81d1, scope.project.id=cee19bd514184a9988cbe093840d4283, user.id=f7fc4b88abc344d2b72a4653dc811819) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:25.615480 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd1779-78aa-4a41-96fd-76fd1f34460d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:25.616933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dd1779-78aa-4a41-96fd-76fd1f34460d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:25.623913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 51/97] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:14:25 2022] GET /identity/v3/role_assignments?user.id=f7fc4b88abc344d2b72a4653dc811819&scope.project.id=cee19bd514184a9988cbe093840d4283&role.id=4912a35790fe4d4297540156083d81d1 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:27.505536 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f487846-6320-419c-92fe-6687ebf8e01c None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:27.506007 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f487846-6320-419c-92fe-6687ebf8e01c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:27.506007 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f487846-6320-419c-92fe-6687ebf8e01c None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:27.507403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 47/98] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:27.520907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6dfa5-852b-4269-9952-df87582d7089 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:27.521385 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6dfa5-852b-4269-9952-df87582d7089 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:27.521621 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6dfa5-852b-4269-9952-df87582d7089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:27.556321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6ed6dfa5-852b-4269-9952-df87582d7089 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:27.557571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6ed6dfa5-852b-4269-9952-df87582d7089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:27.618961 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:27.649790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 52/99] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:14:27.652396 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6bba0-4054-4b89-8334-efc01b6b5d08 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:27.653313 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6bba0-4054-4b89-8334-efc01b6b5d08 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:27.653788 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6bba0-4054-4b89-8334-efc01b6b5d08 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:27.655149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 48/100] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:27.660708 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6742fb-8a56-4c8a-b960-238668691e68 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:27.661005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6742fb-8a56-4c8a-b960-238668691e68 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:27.661291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6742fb-8a56-4c8a-b960-238668691e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:27.682356 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6bba0-4054-4b89-8334-efc01b6b5d08 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:27.690838 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ce6742fb-8a56-4c8a-b960-238668691e68 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:27.692047 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ce6742fb-8a56-4c8a-b960-238668691e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:27.847813 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 53/101] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:14:27.849128 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eea146-d30c-4c1b-9350-058009653396 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:27.849426 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eea146-d30c-4c1b-9350-058009653396 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:27.853785 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eea146-d30c-4c1b-9350-058009653396 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:27.855096 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 54/102] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:27.868368 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6bba0-4054-4b89-8334-efc01b6b5d08 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:27.875124 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ce2a4-428d-4191-b510-1e2b39d6fc51 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:27.875421 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ce2a4-428d-4191-b510-1e2b39d6fc51 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:27.875707 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ce2a4-428d-4191-b510-1e2b39d6fc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:27.906917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5f7ce2a4-428d-4191-b510-1e2b39d6fc51 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:27.908342 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5f7ce2a4-428d-4191-b510-1e2b39d6fc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:27.925111 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8a284b-41b6-49ba-af34-5726676d330f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:27.926222 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a284b-41b6-49ba-af34-5726676d330f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:27.926730 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a284b-41b6-49ba-af34-5726676d330f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:27.927306 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a284b-41b6-49ba-af34-5726676d330f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:27.938854 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8a284b-41b6-49ba-af34-5726676d330f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:27.945082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ac8a284b-41b6-49ba-af34-5726676d330f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:14:27.945082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:27.945082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:27.945082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:27.945082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:27.945082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:27.945082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:27.945082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:27.945082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:27.945082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:27.945082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:27.945082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:27.945082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:27.945082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:27.945082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:27.945082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:27.945963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:27.945963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:27.945963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:27.945963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:27.945963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:27.945963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:27.945963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:27.945963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:27.945963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:27.945963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:27.945963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:27.945963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:27.945963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:27.945963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:27.945963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:27.945963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:27.946715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:27.946715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:27.946715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:27.946715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:27.946715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:27.946715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:27.946715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:27.946715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:27.946715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:27.946715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:27.946715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:14:27.946715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:27.949200 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 49/103] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:14:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:27.955034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e439ca-57b1-4a13-a8f2-aa1f4e5e080a None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:27.955550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e439ca-57b1-4a13-a8f2-aa1f4e5e080a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:27.956023 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e439ca-57b1-4a13-a8f2-aa1f4e5e080a None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:27.957626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 50/104] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:27.958678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e439ca-57b1-4a13-a8f2-aa1f4e5e080a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:27.994671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 55/105] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 28 22:14:27.995668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ce2a4-428d-4191-b510-1e2b39d6fc51 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:28.037152 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:28.049211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:28.118021 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e439ca-57b1-4a13-a8f2-aa1f4e5e080a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:28.125125 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ce2a4-428d-4191-b510-1e2b39d6fc51 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:28.169073 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a1dc8-b869-409d-a80b-155a457fe50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:28.170185 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a1dc8-b869-409d-a80b-155a457fe50f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:28.170721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a1dc8-b869-409d-a80b-155a457fe50f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:28.171176 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a1dc8-b869-409d-a80b-155a457fe50f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:28.178559 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b4c99-b437-4a23-bd60-7d4fbbd49285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:28.179453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b4c99-b437-4a23-bd60-7d4fbbd49285 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:28.180118 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b4c99-b437-4a23-bd60-7d4fbbd49285 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:28.180870 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b4c99-b437-4a23-bd60-7d4fbbd49285 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:28.181845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b4c99-b437-4a23-bd60-7d4fbbd49285 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:28.182353 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b4c99-b437-4a23-bd60-7d4fbbd49285 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:28.183593 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a1dc8-b869-409d-a80b-155a457fe50f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:28.185238 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b4c99-b437-4a23-bd60-7d4fbbd49285 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:28.189941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-787a1dc8-b869-409d-a80b-155a457fe50f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 22:14:28.189941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:28.189941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:28.189941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:28.189941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:28.189941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:28.189941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:28.189941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:28.189941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:28.189941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:28.189941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:28.189941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:28.189941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:28.189941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:28.189941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:28.189941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:28.190824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:28.190824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:28.190824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:28.190824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:28.190824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:28.190824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:28.190824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:28.190824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:28.190824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:28.190824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:28.190824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:28.190824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:28.190824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:28.190824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:28.190824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:28.190824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:28.191616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:28.191616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:28.191616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:28.191616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:28.191616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:28.191616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:28.191616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:28.191616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:28.191616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:28.191616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:28.191616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 22:14:28.191616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:28.191616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 51/106] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:28.194175 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370416dd-300d-426b-b043-a1d3ba76cbb6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:28.194470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370416dd-300d-426b-b043-a1d3ba76cbb6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:28.194751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370416dd-300d-426b-b043-a1d3ba76cbb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:28.198641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 56/107] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:28.200366 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b4c99-b437-4a23-bd60-7d4fbbd49285 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:28.214836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-370416dd-300d-426b-b043-a1d3ba76cbb6 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:28.216899 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-370416dd-300d-426b-b043-a1d3ba76cbb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:28.289575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 52/108] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:28.290888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc117e07-01bd-4ab8-9536-ae8e1c07298e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:28.291184 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc117e07-01bd-4ab8-9536-ae8e1c07298e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:28.291455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc117e07-01bd-4ab8-9536-ae8e1c07298e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:28.296664 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 53/109] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:28.296664 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc117e07-01bd-4ab8-9536-ae8e1c07298e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:28.316524 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b4c99-b437-4a23-bd60-7d4fbbd49285 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:28.318116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:28.336956 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:28.364341 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a33ea-b87b-4a80-9694-c327646e1c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:28.365004 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a33ea-b87b-4a80-9694-c327646e1c9d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:28.365554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a33ea-b87b-4a80-9694-c327646e1c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:28.365846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a33ea-b87b-4a80-9694-c327646e1c9d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:28.367672 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a33ea-b87b-4a80-9694-c327646e1c9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:28.369859 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02a33ea-b87b-4a80-9694-c327646e1c9d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:28.372495 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 57/110] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:14:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:28.373944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a33ea-b87b-4a80-9694-c327646e1c9d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:28.420885 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc117e07-01bd-4ab8-9536-ae8e1c07298e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:28.458554 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a4874-7777-461c-b42f-e3b902a5e66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:28.459181 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a4874-7777-461c-b42f-e3b902a5e66f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:28.459462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a4874-7777-461c-b42f-e3b902a5e66f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:28.459778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a4874-7777-461c-b42f-e3b902a5e66f None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:28.460447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a4874-7777-461c-b42f-e3b902a5e66f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:28.460930 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a4874-7777-461c-b42f-e3b902a5e66f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:28.462179 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a4874-7777-461c-b42f-e3b902a5e66f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:28.476075 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 54/111] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:28.476075 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142880e-e47b-4e5e-8d35-6a9eae865609 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:28.476075 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142880e-e47b-4e5e-8d35-6a9eae865609 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:28.476075 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142880e-e47b-4e5e-8d35-6a9eae865609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:28.486134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02a33ea-b87b-4a80-9694-c327646e1c9d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:28.492630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2142880e-e47b-4e5e-8d35-6a9eae865609 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:28.493829 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2142880e-e47b-4e5e-8d35-6a9eae865609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:28.522179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb4230-f18c-4681-a88f-c2270f640868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:28.522834 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb4230-f18c-4681-a88f-c2270f640868 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:28.523121 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb4230-f18c-4681-a88f-c2270f640868 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:28.523401 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb4230-f18c-4681-a88f-c2270f640868 None admin] PATH_INFO: `/v3/users/0b20c2888f9642efb7858178880ed9a0` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:28.524584 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb4230-f18c-4681-a88f-c2270f640868 None admin] RBAC: Authorizing `identity:get_user(user_id=0b20c2888f9642efb7858178880ed9a0)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:28.526081 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ceb4230-f18c-4681-a88f-c2270f640868 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:28.528221 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 58/112] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:28 2022] GET /identity/v3/users/0b20c2888f9642efb7858178880ed9a0 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:28.528994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb4230-f18c-4681-a88f-c2270f640868 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:28.570761 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 55/113] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:28.571285 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:28.572246 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142880e-e47b-4e5e-8d35-6a9eae865609 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:28.597206 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:28.639301 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb4230-f18c-4681-a88f-c2270f640868 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:28.673102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142880e-e47b-4e5e-8d35-6a9eae865609 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:28.673913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d1933-5da8-4fc8-88cb-2cf8a25cbe2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:28.674561 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d1933-5da8-4fc8-88cb-2cf8a25cbe2f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:28.674846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d1933-5da8-4fc8-88cb-2cf8a25cbe2f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:28.675126 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d1933-5da8-4fc8-88cb-2cf8a25cbe2f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:28.684914 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d1933-5da8-4fc8-88cb-2cf8a25cbe2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:28.689894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-068d1933-5da8-4fc8-88cb-2cf8a25cbe2f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:14:28.689894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:28.689894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:28.689894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:28.689894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:28.689894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:28.689894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:28.689894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:28.689894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:28.689894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:28.689894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:28.689894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:28.689894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:28.689894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:28.689894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:28.689894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:28.690945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:28.690945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:28.690945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:28.690945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:28.690945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:28.690945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:28.690945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:28.690945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:28.690945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:28.690945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:28.690945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:28.690945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:28.690945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:28.690945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:28.690945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:28.690945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:28.691907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:28.691907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:28.691907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:28.691907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:28.691907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:28.691907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:28.691907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:28.691907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:28.691907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:28.691907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:28.691907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:14:28.691907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:14:28.691907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 59/114] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:14:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:28.691907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d1933-5da8-4fc8-88cb-2cf8a25cbe2f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:28.710879 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb35e51-06df-41dd-b980-1e031a01dce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:28.711508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb35e51-06df-41dd-b980-1e031a01dce5 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:28.711797 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb35e51-06df-41dd-b980-1e031a01dce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:28.712081 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb35e51-06df-41dd-b980-1e031a01dce5 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:28.712766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb35e51-06df-41dd-b980-1e031a01dce5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:28.714261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb35e51-06df-41dd-b980-1e031a01dce5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:28.740536 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 56/115] 10.222.0.39 () {60 vars in 1141 bytes} [Thu Jul 28 22:14:28 2022] POST /identity/v3/projects => generated 302 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:14:28.741356 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb35e51-06df-41dd-b980-1e031a01dce5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:28.807978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:28.814846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d1933-5da8-4fc8-88cb-2cf8a25cbe2f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:28.830049 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:28.858408 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb35e51-06df-41dd-b980-1e031a01dce5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:28.859523 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d9bc0-53b5-4cd6-ac0f-ad89d6f885c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:28.860324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d9bc0-53b5-4cd6-ac0f-ad89d6f885c4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:28.860611 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d9bc0-53b5-4cd6-ac0f-ad89d6f885c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:28.861004 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d9bc0-53b5-4cd6-ac0f-ad89d6f885c4 None admin] PATH_INFO: `/v3/users/ce54aa78635b42ed9decf8c08ae6465f` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:28.879351 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d9bc0-53b5-4cd6-ac0f-ad89d6f885c4 None admin] RBAC: Authorizing `identity:get_user(user_id=ce54aa78635b42ed9decf8c08ae6465f)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:28.880839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d9bc0-53b5-4cd6-ac0f-ad89d6f885c4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:28.892533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 60/116] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:28 2022] GET /identity/v3/users/ce54aa78635b42ed9decf8c08ae6465f => generated 289 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:28.892533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78235720-5500-434c-9df6-937175fffcbd None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:28.892533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78235720-5500-434c-9df6-937175fffcbd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:28.892533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78235720-5500-434c-9df6-937175fffcbd None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:28.892533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 61/117] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:28.892533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78235720-5500-434c-9df6-937175fffcbd None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:28.909719 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bfd32-487b-4277-aacd-a4ba38ebe2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:28.910370 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bfd32-487b-4277-aacd-a4ba38ebe2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:28.910651 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bfd32-487b-4277-aacd-a4ba38ebe2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:28.910928 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bfd32-487b-4277-aacd-a4ba38ebe2e1 None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:28.912190 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471bfd32-487b-4277-aacd-a4ba38ebe2e1 None admin] RBAC: Authorizing `identity:get_project(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:28.913691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471bfd32-487b-4277-aacd-a4ba38ebe2e1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:28.915373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 57/118] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:28 2022] GET /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:28.921028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bfd32-487b-4277-aacd-a4ba38ebe2e1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:29.001899 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78235720-5500-434c-9df6-937175fffcbd None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:29.034041 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bfd32-487b-4277-aacd-a4ba38ebe2e1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:29.044164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1369748-6465-42a1-9263-a4088b75990d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:29.044916 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1369748-6465-42a1-9263-a4088b75990d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:29.045200 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1369748-6465-42a1-9263-a4088b75990d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:29.045476 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1369748-6465-42a1-9263-a4088b75990d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:29.055670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1369748-6465-42a1-9263-a4088b75990d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:29.061374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-a1369748-6465-42a1-9263-a4088b75990d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:29.061374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:29.061374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:29.061374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:29.061374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:29.061374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:29.061374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:29.061374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:29.061374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:29.061374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:29.061374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:29.061374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:29.061374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:29.061374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:29.061374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:29.061374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:29.062459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:29.062459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:29.062459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:29.062459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:29.062459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:29.062459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:29.062459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:29.062459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:29.062459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:29.062459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:29.062459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:29.062459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:29.062459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:29.062459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:29.062459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:29.062459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:29.063440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:29.063440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:29.063440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:29.063440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:29.063440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:29.063440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:29.063440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:29.063440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:29.063440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:29.063440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:29.063440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:29.063440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:14:29.063440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 62/119] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:29.063440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1369748-6465-42a1-9263-a4088b75990d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:29.076304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc88d71-040b-412d-88f1-554801bc03c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:29.077047 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc88d71-040b-412d-88f1-554801bc03c3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:29.077403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc88d71-040b-412d-88f1-554801bc03c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:29.077772 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc88d71-040b-412d-88f1-554801bc03c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:29.078584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc88d71-040b-412d-88f1-554801bc03c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:29.078963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc88d71-040b-412d-88f1-554801bc03c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:29.080409 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc88d71-040b-412d-88f1-554801bc03c3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:29.084832 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:29.089491 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 58/120] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:14:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:29.091717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e7720-900f-4d75-a86f-ad576cd7eca8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:29.092091 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e7720-900f-4d75-a86f-ad576cd7eca8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:29.092441 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e7720-900f-4d75-a86f-ad576cd7eca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:29.099926 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:29.111962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-503e7720-900f-4d75-a86f-ad576cd7eca8 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:29.113174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-503e7720-900f-4d75-a86f-ad576cd7eca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:29.177392 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1369748-6465-42a1-9263-a4088b75990d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:29.190876 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 59/121] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:29.194650 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e7720-900f-4d75-a86f-ad576cd7eca8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:29.214477 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ecb75-b923-4001-9e82-5dcbad8301c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:29.216163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ecb75-b923-4001-9e82-5dcbad8301c1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:29.216454 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ecb75-b923-4001-9e82-5dcbad8301c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:29.216731 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ecb75-b923-4001-9e82-5dcbad8301c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:29.287654 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ecb75-b923-4001-9e82-5dcbad8301c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:29.289124 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758ecb75-b923-4001-9e82-5dcbad8301c1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:29.305973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:29.309921 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e7720-900f-4d75-a86f-ad576cd7eca8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:29.348149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8470e8-15e0-4376-8cf1-72630e35c952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:29.349127 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8470e8-15e0-4376-8cf1-72630e35c952 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:29.349627 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8470e8-15e0-4376-8cf1-72630e35c952 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:29.350169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8470e8-15e0-4376-8cf1-72630e35c952 None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29/users/0b20c2888f9642efb7858178880ed9a0/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:29.352367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8470e8-15e0-4376-8cf1-72630e35c952 None admin] RBAC: Authorizing `identity:create_grant(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29, user_id=0b20c2888f9642efb7858178880ed9a0, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:29.354212 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8470e8-15e0-4376-8cf1-72630e35c952 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:29.356882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:29.368068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 60/122] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:14:29 2022] PUT /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29/users/0b20c2888f9642efb7858178880ed9a0/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:14:29.369272 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8470e8-15e0-4376-8cf1-72630e35c952 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:29.411773 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 63/123] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:14:29.412940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ecb75-b923-4001-9e82-5dcbad8301c1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:29.508824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8470e8-15e0-4376-8cf1-72630e35c952 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:29.550863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7883a-fcdc-4cf6-b762-25dbabd62e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:29.551459 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7883a-fcdc-4cf6-b762-25dbabd62e69 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:29.551748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7883a-fcdc-4cf6-b762-25dbabd62e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:29.552024 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7883a-fcdc-4cf6-b762-25dbabd62e69 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:29.552752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7883a-fcdc-4cf6-b762-25dbabd62e69 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:29.553830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7883a-fcdc-4cf6-b762-25dbabd62e69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:29.555833 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7883a-fcdc-4cf6-b762-25dbabd62e69 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:29.564104 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 61/124] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:29.564337 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ecb75-b923-4001-9e82-5dcbad8301c1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:29.564958 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7883a-fcdc-4cf6-b762-25dbabd62e69 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:29.586863 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:29.613196 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd39818-4388-43bc-a463-db607aff3cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:29.614769 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd39818-4388-43bc-a463-db607aff3cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:29.615273 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd39818-4388-43bc-a463-db607aff3cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:29.615777 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd39818-4388-43bc-a463-db607aff3cd0 None admin] PATH_INFO: `/v3/users/f7fc4b88abc344d2b72a4653dc811819` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:29.617705 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd39818-4388-43bc-a463-db607aff3cd0 None admin] RBAC: Authorizing `identity:get_user(user_id=f7fc4b88abc344d2b72a4653dc811819)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:29.620112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd39818-4388-43bc-a463-db607aff3cd0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:29.623339 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 64/125] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:29 2022] GET /identity/v3/users/f7fc4b88abc344d2b72a4653dc811819 => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:29.624320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd39818-4388-43bc-a463-db607aff3cd0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:29.632903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:29.685385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7883a-fcdc-4cf6-b762-25dbabd62e69 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:29.727971 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085b0f9-3156-4245-bae8-35e970a386b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:29.728930 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085b0f9-3156-4245-bae8-35e970a386b4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:29.729445 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085b0f9-3156-4245-bae8-35e970a386b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:29.729954 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7085b0f9-3156-4245-bae8-35e970a386b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:29.745170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7085b0f9-3156-4245-bae8-35e970a386b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:29.753962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7085b0f9-3156-4245-bae8-35e970a386b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:29.753962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:29.753962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:29.753962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:29.753962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:29.753962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:29.753962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:29.753962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:29.753962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:29.753962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:29.753962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:29.753962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:29.753962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:29.753962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:29.753962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:29.753962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:29.754841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:29.754841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:29.754841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:29.754841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:29.754841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:29.754841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:29.754841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:29.754841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:29.754841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:29.754841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:29.754841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:29.754841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:29.754841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:29.754841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:29.754841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:29.754841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:29.755609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:29.755609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:29.755609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:29.755609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:29.755609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:29.755609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:29.755609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:29.755609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:29.755609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:29.755609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:29.755609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:29.755609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:29.758380 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 62/126] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:29.759314 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085b0f9-3156-4245-bae8-35e970a386b4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:29.768903 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd39818-4388-43bc-a463-db607aff3cd0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:29.805505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a914695-595e-4e6b-99d1-7ed3796cfd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:29.806376 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a914695-595e-4e6b-99d1-7ed3796cfd9e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:29.806759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a914695-595e-4e6b-99d1-7ed3796cfd9e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:29.807135 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a914695-595e-4e6b-99d1-7ed3796cfd9e None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:29.808528 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a914695-595e-4e6b-99d1-7ed3796cfd9e None admin] RBAC: Authorizing `identity:get_project(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:29.810338 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a914695-595e-4e6b-99d1-7ed3796cfd9e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:29.812488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 65/127] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:29 2022] GET /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29 => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:29.813437 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a914695-595e-4e6b-99d1-7ed3796cfd9e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:29.870694 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:29.903265 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7085b0f9-3156-4245-bae8-35e970a386b4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:29.920708 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:29.924197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a914695-595e-4e6b-99d1-7ed3796cfd9e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:29.941434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f185cd1-341c-4d66-93dd-b9dda5c44b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:29.942329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f185cd1-341c-4d66-93dd-b9dda5c44b99 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:29.942763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f185cd1-341c-4d66-93dd-b9dda5c44b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:29.943188 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f185cd1-341c-4d66-93dd-b9dda5c44b99 None admin] PATH_INFO: `/v3/users/f7fc4b88abc344d2b72a4653dc811819` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:29.944550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f185cd1-341c-4d66-93dd-b9dda5c44b99 None admin] RBAC: Authorizing `identity:get_user(user_id=f7fc4b88abc344d2b72a4653dc811819)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:29.946214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f185cd1-341c-4d66-93dd-b9dda5c44b99 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:29.947981 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 63/128] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:29 2022] GET /identity/v3/users/f7fc4b88abc344d2b72a4653dc811819 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:29.953868 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f185cd1-341c-4d66-93dd-b9dda5c44b99 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:29.961724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8af8f-9651-4f80-bb04-4e2b4c8282b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:29.962380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8af8f-9651-4f80-bb04-4e2b4c8282b5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:29.962731 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8af8f-9651-4f80-bb04-4e2b4c8282b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:29.963020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc8af8f-9651-4f80-bb04-4e2b4c8282b5 None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:29.964209 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8af8f-9651-4f80-bb04-4e2b4c8282b5 None admin] RBAC: Authorizing `identity:get_project(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:29.965809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc8af8f-9651-4f80-bb04-4e2b4c8282b5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:29.967708 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 66/129] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:29 2022] GET /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29 => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:29.968271 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8af8f-9651-4f80-bb04-4e2b4c8282b5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:30.062134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f185cd1-341c-4d66-93dd-b9dda5c44b99 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:30.072193 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc8af8f-9651-4f80-bb04-4e2b4c8282b5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:30.093654 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:30.099376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277efb2-929b-441d-8d79-32e0924123e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:30.100192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277efb2-929b-441d-8d79-32e0924123e4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:30.100625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277efb2-929b-441d-8d79-32e0924123e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:30.101048 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4277efb2-929b-441d-8d79-32e0924123e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:30.101984 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277efb2-929b-441d-8d79-32e0924123e4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:30.102521 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277efb2-929b-441d-8d79-32e0924123e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:30.104293 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4277efb2-929b-441d-8d79-32e0924123e4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:30.109774 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf7e2d-aca1-4a90-869a-734bc4119002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:30.110401 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf7e2d-aca1-4a90-869a-734bc4119002 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:30.110690 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf7e2d-aca1-4a90-869a-734bc4119002 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:30.110971 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bf7e2d-aca1-4a90-869a-734bc4119002 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:30.111768 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf7e2d-aca1-4a90-869a-734bc4119002 None admin] RBAC: Adding query filter params (role.id=d2fa7787cf3f461589be9f37afe9de6e, scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29, user.id=ce54aa78635b42ed9decf8c08ae6465f) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:30.112087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf7e2d-aca1-4a90-869a-734bc4119002 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:30.113562 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bf7e2d-aca1-4a90-869a-734bc4119002 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:30.115891 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 64/130] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:30.116932 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277efb2-929b-441d-8d79-32e0924123e4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:30.119416 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 67/131] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:14:29 2022] GET /identity/v3/role_assignments?user.id=ce54aa78635b42ed9decf8c08ae6465f&scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29&role.id=d2fa7787cf3f461589be9f37afe9de6e => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:30.120137 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf7e2d-aca1-4a90-869a-734bc4119002 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:30.172153 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:30.232014 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bf7e2d-aca1-4a90-869a-734bc4119002 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:30.232351 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4277efb2-929b-441d-8d79-32e0924123e4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:30.279289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a04496-faff-4c3d-90e9-f7032f6bf5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:30.279986 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a04496-faff-4c3d-90e9-f7032f6bf5a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:30.280276 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a04496-faff-4c3d-90e9-f7032f6bf5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:30.280561 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a04496-faff-4c3d-90e9-f7032f6bf5a3 None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29/users/f7fc4b88abc344d2b72a4653dc811819/roles/4146966c23674402aebb9a5b21cf18ba` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:30.284734 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a04496-faff-4c3d-90e9-f7032f6bf5a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29, user_id=f7fc4b88abc344d2b72a4653dc811819, role_id=4146966c23674402aebb9a5b21cf18ba)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:30.286473 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a04496-faff-4c3d-90e9-f7032f6bf5a3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:30.302865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909efbd-893b-4ed4-bab7-0d1f29a1c3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:30.303540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909efbd-893b-4ed4-bab7-0d1f29a1c3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:30.303828 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909efbd-893b-4ed4-bab7-0d1f29a1c3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:30.304112 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a909efbd-893b-4ed4-bab7-0d1f29a1c3e3 None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:30.305670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a909efbd-893b-4ed4-bab7-0d1f29a1c3e3 None admin] RBAC: Authorizing `identity:get_project(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:30.307152 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a909efbd-893b-4ed4-bab7-0d1f29a1c3e3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:30.309461 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 65/132] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:30 2022] GET /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:30.313076 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909efbd-893b-4ed4-bab7-0d1f29a1c3e3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:30.317086 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 68/133] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:14:30 2022] PUT /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29/users/f7fc4b88abc344d2b72a4653dc811819/roles/4146966c23674402aebb9a5b21cf18ba => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:14:30.317940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a04496-faff-4c3d-90e9-f7032f6bf5a3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:30.412525 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:30.527207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a909efbd-893b-4ed4-bab7-0d1f29a1c3e3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:30.530192 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a04496-faff-4c3d-90e9-f7032f6bf5a3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:30.558910 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:30.601357 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b7155-e570-4e89-868d-a11557125005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:30.602094 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b7155-e570-4e89-868d-a11557125005 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:30.602385 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b7155-e570-4e89-868d-a11557125005 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:30.602671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b7155-e570-4e89-868d-a11557125005 None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29/users/f7fc4b88abc344d2b72a4653dc811819/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:30.605803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b7155-e570-4e89-868d-a11557125005 None admin] RBAC: Authorizing `identity:create_grant(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29, user_id=f7fc4b88abc344d2b72a4653dc811819, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:30.607297 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164b7155-e570-4e89-868d-a11557125005 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:30.611080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe3659-31ca-40b7-86ca-e1815e510c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:30.611748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe3659-31ca-40b7-86ca-e1815e510c9f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:30.612140 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe3659-31ca-40b7-86ca-e1815e510c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:30.612422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe3659-31ca-40b7-86ca-e1815e510c9f None admin] PATH_INFO: `/v3/users/f7fc4b88abc344d2b72a4653dc811819` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:30.613700 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe3659-31ca-40b7-86ca-e1815e510c9f None admin] RBAC: Authorizing `identity:get_user(user_id=f7fc4b88abc344d2b72a4653dc811819)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:30.615328 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe3659-31ca-40b7-86ca-e1815e510c9f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:30.617324 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 66/134] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:30 2022] GET /identity/v3/users/f7fc4b88abc344d2b72a4653dc811819 => generated 275 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:30.624135 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe3659-31ca-40b7-86ca-e1815e510c9f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:30.634725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 69/135] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:14:30 2022] PUT /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29/users/f7fc4b88abc344d2b72a4653dc811819/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:14:30.748145 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe3659-31ca-40b7-86ca-e1815e510c9f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:30.795824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83e874-b1e1-4637-a9da-81e582a2043d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:30.796403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83e874-b1e1-4637-a9da-81e582a2043d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:30.796688 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83e874-b1e1-4637-a9da-81e582a2043d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:30.796968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83e874-b1e1-4637-a9da-81e582a2043d None admin] PATH_INFO: `/v3/projects/cee19bd514184a9988cbe093840d4283` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:30.798466 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83e874-b1e1-4637-a9da-81e582a2043d None admin] RBAC: Authorizing `identity:get_project(project_id=cee19bd514184a9988cbe093840d4283)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:30.799942 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83e874-b1e1-4637-a9da-81e582a2043d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:30.803535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 67/136] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:30 2022] GET /identity/v3/projects/cee19bd514184a9988cbe093840d4283 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:30.810471 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b7155-e570-4e89-868d-a11557125005 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:30.956378 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b7155-e570-4e89-868d-a11557125005 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:30.958847 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:30.993432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1321a-c3d9-4f13-9266-eb9eefaf0771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:30.994184 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1321a-c3d9-4f13-9266-eb9eefaf0771 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:30.994473 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1321a-c3d9-4f13-9266-eb9eefaf0771 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:30.994746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1321a-c3d9-4f13-9266-eb9eefaf0771 None admin] PATH_INFO: `/v3/projects/cee19bd514184a9988cbe093840d4283/users/f7fc4b88abc344d2b72a4653dc811819/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:30.997388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1321a-c3d9-4f13-9266-eb9eefaf0771 None admin] RBAC: Authorizing `identity:create_grant(project_id=cee19bd514184a9988cbe093840d4283, user_id=f7fc4b88abc344d2b72a4653dc811819, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:30.999103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1321a-c3d9-4f13-9266-eb9eefaf0771 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:31.015774 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 70/137] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:14:30 2022] PUT /identity/v3/projects/cee19bd514184a9988cbe093840d4283/users/f7fc4b88abc344d2b72a4653dc811819/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:14:31.122911 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f81aa8-f1a1-4923-bd7a-15f7c77735e1 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:31.123237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f81aa8-f1a1-4923-bd7a-15f7c77735e1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:31.123491 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f81aa8-f1a1-4923-bd7a-15f7c77735e1 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:31.125165 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 68/138] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:31.130604 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a9cf3-1e64-4659-a7d4-2585bc50d61a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:31.130905 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a9cf3-1e64-4659-a7d4-2585bc50d61a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:31.131187 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a9cf3-1e64-4659-a7d4-2585bc50d61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:31.171151 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-707a9cf3-1e64-4659-a7d4-2585bc50d61a None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:31.172370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-707a9cf3-1e64-4659-a7d4-2585bc50d61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:31.253184 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 71/139] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:31.290001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f81aa8-f1a1-4923-bd7a-15f7c77735e1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:31.362170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:31.419703 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f81aa8-f1a1-4923-bd7a-15f7c77735e1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:31.455803 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644d3b2-ea2f-4fb0-9eee-47e00851d7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:31.456506 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644d3b2-ea2f-4fb0-9eee-47e00851d7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:31.456797 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644d3b2-ea2f-4fb0-9eee-47e00851d7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:31.457075 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644d3b2-ea2f-4fb0-9eee-47e00851d7f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:31.458272 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644d3b2-ea2f-4fb0-9eee-47e00851d7f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:31.459747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644d3b2-ea2f-4fb0-9eee-47e00851d7f9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:31.461594 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 69/140] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:14:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:31.467782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a9cf3-1e64-4659-a7d4-2585bc50d61a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:31.612413 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:31.619259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a9cf3-1e64-4659-a7d4-2585bc50d61a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:31.657374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489f3b3-fa5d-4fe1-98d8-e938524b54ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:31.658040 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489f3b3-fa5d-4fe1-98d8-e938524b54ce None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:31.658321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489f3b3-fa5d-4fe1-98d8-e938524b54ce None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:31.658597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489f3b3-fa5d-4fe1-98d8-e938524b54ce None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:31.659305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489f3b3-fa5d-4fe1-98d8-e938524b54ce None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:31.660770 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489f3b3-fa5d-4fe1-98d8-e938524b54ce None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:31.705868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 72/141] 10.222.0.39 () {60 vars in 1132 bytes} [Thu Jul 28 22:14:31 2022] POST /identity/v3/users => generated 283 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:14:31.926466 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28173a-b3a9-4c44-b243-bf6bf3c73824 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:31.927520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28173a-b3a9-4c44-b243-bf6bf3c73824 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:31.928148 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28173a-b3a9-4c44-b243-bf6bf3c73824 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:31.929818 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 70/142] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:31.936152 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d768a41-0c65-404c-8ef8-31cce97753a1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:31.936442 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d768a41-0c65-404c-8ef8-31cce97753a1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:31.936722 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d768a41-0c65-404c-8ef8-31cce97753a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:31.974343 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3d768a41-0c65-404c-8ef8-31cce97753a1 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:31.975468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3d768a41-0c65-404c-8ef8-31cce97753a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:32.068365 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 73/143] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:32.104343 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28173a-b3a9-4c44-b243-bf6bf3c73824 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:32.243007 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28173a-b3a9-4c44-b243-bf6bf3c73824 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:32.267002 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:32.292249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314a077-4588-491a-bbc8-6174734af7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:32.294198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314a077-4588-491a-bbc8-6174734af7dd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:32.294900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314a077-4588-491a-bbc8-6174734af7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:32.295556 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b314a077-4588-491a-bbc8-6174734af7dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:32.319158 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b314a077-4588-491a-bbc8-6174734af7dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:32.326192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-b314a077-4588-491a-bbc8-6174734af7dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:14:32.326192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:32.326192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:32.326192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:32.326192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:32.326192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:32.326192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:32.326192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:32.326192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:32.326192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:32.326192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:32.326192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:32.326192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:32.326192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:32.326192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:32.326192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:32.327540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:32.327540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:32.327540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:32.327540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:32.327540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:32.327540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:32.327540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:32.327540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:32.327540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:32.327540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:32.327540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:32.327540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:32.327540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:32.327540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:32.327540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:32.327540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:32.328342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:32.328342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:32.328342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:32.328342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:32.328342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:32.328342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:32.328342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:32.328342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:32.328342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:32.328342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:32.328342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:14:32.328342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:32.331270 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 71/144] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:14:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:32.337372 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d768a41-0c65-404c-8ef8-31cce97753a1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:32.436822 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:32.458259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d768a41-0c65-404c-8ef8-31cce97753a1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:32.503734 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e29d4-7ba0-4912-85c2-ffecdbfe646d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:32.504278 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e29d4-7ba0-4912-85c2-ffecdbfe646d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:32.504560 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e29d4-7ba0-4912-85c2-ffecdbfe646d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:32.504835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e29d4-7ba0-4912-85c2-ffecdbfe646d None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:32.505578 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e29d4-7ba0-4912-85c2-ffecdbfe646d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:32.505885 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e29d4-7ba0-4912-85c2-ffecdbfe646d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:32.507269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e29d4-7ba0-4912-85c2-ffecdbfe646d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:32.520619 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 74/145] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:32.526150 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314a077-4588-491a-bbc8-6174734af7dd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:32.650088 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b314a077-4588-491a-bbc8-6174734af7dd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:32.701031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eab3c2-7db3-4f7f-ac2d-fb02aff24e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:32.701767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eab3c2-7db3-4f7f-ac2d-fb02aff24e86 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:32.702053 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eab3c2-7db3-4f7f-ac2d-fb02aff24e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:32.702327 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eab3c2-7db3-4f7f-ac2d-fb02aff24e86 None admin] PATH_INFO: `/v3/users/0b20c2888f9642efb7858178880ed9a0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:32.706243 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eab3c2-7db3-4f7f-ac2d-fb02aff24e86 None admin] RBAC: Authorizing `identity:get_user(user_id=0b20c2888f9642efb7858178880ed9a0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:32.707739 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eab3c2-7db3-4f7f-ac2d-fb02aff24e86 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:32.709559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 72/146] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:32 2022] GET /identity/v3/users/0b20c2888f9642efb7858178880ed9a0 => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:32.715003 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e29d4-7ba0-4912-85c2-ffecdbfe646d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:32.850155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e29d4-7ba0-4912-85c2-ffecdbfe646d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:32.889331 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebaf71b-2f3c-457a-8152-4d7520fe80a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:32.889993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebaf71b-2f3c-457a-8152-4d7520fe80a6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:32.890529 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebaf71b-2f3c-457a-8152-4d7520fe80a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:32.890807 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebaf71b-2f3c-457a-8152-4d7520fe80a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:32.907619 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:32.971526 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebaf71b-2f3c-457a-8152-4d7520fe80a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:32.973011 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebaf71b-2f3c-457a-8152-4d7520fe80a6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:33.048678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04290948-ebf2-4d67-a95b-4dc92e6e3f5f None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:33.049194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04290948-ebf2-4d67-a95b-4dc92e6e3f5f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:33.049194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04290948-ebf2-4d67-a95b-4dc92e6e3f5f None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:33.050744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 73/147] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:33.056376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f4938-08d3-431b-a5a9-20a8a63b9995 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:33.056661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f4938-08d3-431b-a5a9-20a8a63b9995 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:33.056940 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f4938-08d3-431b-a5a9-20a8a63b9995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:33.084531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-101f4938-08d3-431b-a5a9-20a8a63b9995 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:33.086108 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-101f4938-08d3-431b-a5a9-20a8a63b9995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:33.087193 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 75/148] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:14:33.094806 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebaf71b-2f3c-457a-8152-4d7520fe80a6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:33.167967 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:33.191152 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:33.256171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebaf71b-2f3c-457a-8152-4d7520fe80a6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:33.271257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 74/149] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:14:33.296351 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c5279-f288-4041-80d9-5fd6b9ac96be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:33.297168 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c5279-f288-4041-80d9-5fd6b9ac96be None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:33.297602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c5279-f288-4041-80d9-5fd6b9ac96be None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:33.297985 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c5279-f288-4041-80d9-5fd6b9ac96be None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:33.301094 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c5279-f288-4041-80d9-5fd6b9ac96be None admin] RBAC: Authorizing `identity:get_project(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:33.302712 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21c5279-f288-4041-80d9-5fd6b9ac96be None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:33.304604 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 76/150] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:33 2022] GET /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:33.311041 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f4938-08d3-431b-a5a9-20a8a63b9995 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:33.318969 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c5279-f288-4041-80d9-5fd6b9ac96be None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:33.473650 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21c5279-f288-4041-80d9-5fd6b9ac96be None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:33.493946 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:33.511714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f4938-08d3-431b-a5a9-20a8a63b9995 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:33.514653 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8f32f-4aa7-4f5e-9cb1-591debe3f082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:33.515565 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8f32f-4aa7-4f5e-9cb1-591debe3f082 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:33.515849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8f32f-4aa7-4f5e-9cb1-591debe3f082 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:33.516125 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8f32f-4aa7-4f5e-9cb1-591debe3f082 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:33.539234 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e8f32f-4aa7-4f5e-9cb1-591debe3f082 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:33.546482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-76e8f32f-4aa7-4f5e-9cb1-591debe3f082 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 22:14:33.546482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:33.546482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:33.546482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:33.546482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:33.546482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:33.546482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:33.546482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:33.546482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:33.546482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:33.546482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:33.546482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:33.546482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:33.546482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:33.546482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:33.546482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:33.547408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:33.547408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:33.547408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:33.547408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:33.547408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:33.547408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:33.547408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:33.547408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:33.547408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:33.547408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:33.547408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:33.547408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:33.547408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:33.547408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:33.547408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:33.547408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:33.548567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:33.548567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:33.548567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:33.548567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:33.548567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:33.548567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:33.548567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:33.548567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:33.548567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:33.548567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:33.548567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 22:14:33.548567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:14:33.548567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 77/151] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:14:33.555246 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8f32f-4aa7-4f5e-9cb1-591debe3f082 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:33.650831 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85841c6d-5437-4240-8010-f2690d5308ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:33.651472 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85841c6d-5437-4240-8010-f2690d5308ea None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:33.651752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85841c6d-5437-4240-8010-f2690d5308ea None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:33.652021 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85841c6d-5437-4240-8010-f2690d5308ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:33.653352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85841c6d-5437-4240-8010-f2690d5308ea None admin] RBAC: Adding query filter params (role.id=aded7bd43e974686b3c23bd9d4255b24, scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29, user.id=0b20c2888f9642efb7858178880ed9a0) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:33.657741 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85841c6d-5437-4240-8010-f2690d5308ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:33.659255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85841c6d-5437-4240-8010-f2690d5308ea None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:33.669905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 75/152] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:14:33 2022] GET /identity/v3/role_assignments?user.id=0b20c2888f9642efb7858178880ed9a0&scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29&role.id=aded7bd43e974686b3c23bd9d4255b24 => generated 608 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:33.706071 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8f32f-4aa7-4f5e-9cb1-591debe3f082 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:33.757832 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bc170-f98d-4de0-acb8-df420c5109e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:33.758449 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bc170-f98d-4de0-acb8-df420c5109e9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:33.758685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bc170-f98d-4de0-acb8-df420c5109e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:33.758926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176bc170-f98d-4de0-acb8-df420c5109e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:33.759671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176bc170-f98d-4de0-acb8-df420c5109e9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:33.759978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176bc170-f98d-4de0-acb8-df420c5109e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:33.761481 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176bc170-f98d-4de0-acb8-df420c5109e9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:33.773654 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 78/153] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:33.779900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85841c6d-5437-4240-8010-f2690d5308ea None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:33.851843 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:33.930140 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85841c6d-5437-4240-8010-f2690d5308ea None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:33.978599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2bbe3-87a0-492f-99d6-9cd8ff0c63e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:33.979252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2bbe3-87a0-492f-99d6-9cd8ff0c63e1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:33.979534 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2bbe3-87a0-492f-99d6-9cd8ff0c63e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:33.979806 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf2bbe3-87a0-492f-99d6-9cd8ff0c63e1 None admin] PATH_INFO: `/v3/users/ce54aa78635b42ed9decf8c08ae6465f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:33.981569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf2bbe3-87a0-492f-99d6-9cd8ff0c63e1 None admin] RBAC: Authorizing `identity:get_user(user_id=ce54aa78635b42ed9decf8c08ae6465f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:33.983014 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf2bbe3-87a0-492f-99d6-9cd8ff0c63e1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:33.984921 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 76/154] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:33 2022] GET /identity/v3/users/ce54aa78635b42ed9decf8c08ae6465f => generated 289 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:33.990485 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bc170-f98d-4de0-acb8-df420c5109e9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:34.040726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:34.119557 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176bc170-f98d-4de0-acb8-df420c5109e9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:34.176951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59f837e-2139-42b0-a710-2b40cd48d966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:34.177641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f837e-2139-42b0-a710-2b40cd48d966 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:34.177926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f837e-2139-42b0-a710-2b40cd48d966 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:34.178203 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f837e-2139-42b0-a710-2b40cd48d966 None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:34.179446 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f837e-2139-42b0-a710-2b40cd48d966 None admin] RBAC: Authorizing `identity:get_project(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:34.180906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59f837e-2139-42b0-a710-2b40cd48d966 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:34.182924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 79/155] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:33 2022] GET /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:34.188416 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2bbe3-87a0-492f-99d6-9cd8ff0c63e1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:34.324853 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf2bbe3-87a0-492f-99d6-9cd8ff0c63e1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:34.334517 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:34.360345 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c9239-1536-4810-87b5-ca331ab7cb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:34.361020 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c9239-1536-4810-87b5-ca331ab7cb5b None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:34.361298 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c9239-1536-4810-87b5-ca331ab7cb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:34.361588 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c9239-1536-4810-87b5-ca331ab7cb5b None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29/users/ce54aa78635b42ed9decf8c08ae6465f/roles/d2fa7787cf3f461589be9f37afe9de6e` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:34.366811 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c9239-1536-4810-87b5-ca331ab7cb5b None admin] RBAC: Authorizing `identity:create_grant(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29, user_id=ce54aa78635b42ed9decf8c08ae6465f, role_id=d2fa7787cf3f461589be9f37afe9de6e)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:34.368284 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c9239-1536-4810-87b5-ca331ab7cb5b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:34.398262 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 77/156] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:14:34 2022] PUT /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29/users/ce54aa78635b42ed9decf8c08ae6465f/roles/d2fa7787cf3f461589be9f37afe9de6e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:14:34.536326 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749aa4f-e026-4912-a69f-8f1456fbb53c None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:34.538469 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749aa4f-e026-4912-a69f-8f1456fbb53c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:34.538469 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749aa4f-e026-4912-a69f-8f1456fbb53c None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:34.538469 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 80/157] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:34.544238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e4ebb-7a01-4782-a4ce-92c36b1d5adf None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:34.544527 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e4ebb-7a01-4782-a4ce-92c36b1d5adf None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:34.544805 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483e4ebb-7a01-4782-a4ce-92c36b1d5adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:34.553222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c4ca9-58b9-4398-a526-4c1556492fc9 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:34.553867 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c4ca9-58b9-4398-a526-4c1556492fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:34.554353 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c4ca9-58b9-4398-a526-4c1556492fc9 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:34.555910 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 81/158] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:14:34.562422 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc673d8a-4e08-48b9-8d38-b938e135f19b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:34.563035 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc673d8a-4e08-48b9-8d38-b938e135f19b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:34.563507 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc673d8a-4e08-48b9-8d38-b938e135f19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:34.582642 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-483e4ebb-7a01-4782-a4ce-92c36b1d5adf None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:34.583753 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-483e4ebb-7a01-4782-a4ce-92c36b1d5adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:34.588374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-cc673d8a-4e08-48b9-8d38-b938e135f19b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:34.590155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-cc673d8a-4e08-48b9-8d38-b938e135f19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:34.711265 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 78/159] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:14:34.731792 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 82/160] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:34.740755 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e4ebb-7a01-4782-a4ce-92c36b1d5adf None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:34.779344 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476ca66-49e2-45f7-b54b-5f1021f8b5c5 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:34.780323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476ca66-49e2-45f7-b54b-5f1021f8b5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:34.780797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476ca66-49e2-45f7-b54b-5f1021f8b5c5 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:34.782470 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 83/161] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:14:34.787773 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624fc09-92c1-4465-866b-b8d4cdaa7b70 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:34.788354 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624fc09-92c1-4465-866b-b8d4cdaa7b70 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:34.788842 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c624fc09-92c1-4465-866b-b8d4cdaa7b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:34.811546 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c624fc09-92c1-4465-866b-b8d4cdaa7b70 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:34.813095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c624fc09-92c1-4465-866b-b8d4cdaa7b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:34.841231 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:34.893985 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:34.898956 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483e4ebb-7a01-4782-a4ce-92c36b1d5adf None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:34.912301 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 84/162] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:34.916386 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624fc09-92c1-4465-866b-b8d4cdaa7b70 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:34.953108 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1672be-cc92-4052-ae79-00227a290880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:34.954382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1672be-cc92-4052-ae79-00227a290880 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:34.954992 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1672be-cc92-4052-ae79-00227a290880 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:34.955628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1672be-cc92-4052-ae79-00227a290880 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:34.989747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1672be-cc92-4052-ae79-00227a290880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:34.996452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-0c1672be-cc92-4052-ae79-00227a290880 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:34.996452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:34.996452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:34.996452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:34.996452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:34.996452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:34.996452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:34.996452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:34.996452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:34.996452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:34.996452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:34.996452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:34.996452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:34.996452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:34.996452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:34.996452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:34.997757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:34.997757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:34.997757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:34.997757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:34.997757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:34.997757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:34.997757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:34.997757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:34.997757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:34.997757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:34.997757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:34.997757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:34.997757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:34.997757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:34.997757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:34.997757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:34.999238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:34.999238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:34.999238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:34.999238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:34.999238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:34.999238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:34.999238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:34.999238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:34.999238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:34.999238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:34.999238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:34.999238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:35.012994 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 79/163] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:35.017354 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1672be-cc92-4052-ae79-00227a290880 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:35.194346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1672be-cc92-4052-ae79-00227a290880 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:35.251730 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc49bf-2963-4af5-8e58-66497c954874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:35.252389 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc49bf-2963-4af5-8e58-66497c954874 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:35.252666 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc49bf-2963-4af5-8e58-66497c954874 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:35.253000 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dc49bf-2963-4af5-8e58-66497c954874 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:35.268768 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dc49bf-2963-4af5-8e58-66497c954874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:35.275283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-46dc49bf-2963-4af5-8e58-66497c954874 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:14:35.275283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:35.275283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:35.275283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:35.275283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:35.275283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:35.275283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:35.275283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:35.275283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:35.275283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:35.275283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:35.275283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:35.275283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:35.275283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:35.275283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:35.275283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:35.276259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:35.276259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:35.276259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:35.276259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:35.276259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:35.276259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:35.276259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:35.276259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:35.276259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:35.276259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:35.276259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:35.276259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:35.276259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:35.276259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:35.276259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:35.276259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:35.277069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:35.277069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:35.277069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:35.277069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:35.277069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:35.277069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:35.277069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:35.277069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:35.277069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:35.277069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:35.277069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:14:35.277069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:35.277069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 80/164] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:14:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:35.278513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc49bf-2963-4af5-8e58-66497c954874 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:35.308267 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c624fc09-92c1-4465-866b-b8d4cdaa7b70 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:35.317883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:35.392856 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91d1f9-6ac5-4b9f-bfc4-08f59f83335b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:35.393510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91d1f9-6ac5-4b9f-bfc4-08f59f83335b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:35.393817 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91d1f9-6ac5-4b9f-bfc4-08f59f83335b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:35.394089 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91d1f9-6ac5-4b9f-bfc4-08f59f83335b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:35.418257 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91d1f9-6ac5-4b9f-bfc4-08f59f83335b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:35.423377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ca91d1f9-6ac5-4b9f-bfc4-08f59f83335b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:35.423377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:35.423377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:35.423377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:35.423377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:35.423377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:35.423377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:35.423377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:35.423377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:35.423377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:35.423377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:35.423377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:35.423377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:35.423377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:35.423377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:35.423377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:35.424467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:35.424467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:35.424467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:35.424467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:35.424467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:35.424467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:35.424467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:35.424467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:35.424467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:35.424467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:35.424467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:35.424467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:35.424467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:35.424467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:35.424467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:35.424467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:35.425642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:35.425642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:35.425642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:35.425642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:35.425642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:35.425642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:35.425642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:35.425642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:35.425642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:35.425642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:35.425642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:35.425642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:14:35.425642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 85/165] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 512 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:35.425642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91d1f9-6ac5-4b9f-bfc4-08f59f83335b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:35.453417 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:35.571489 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dc49bf-2963-4af5-8e58-66497c954874 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:35.583842 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91d1f9-6ac5-4b9f-bfc4-08f59f83335b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:35.648405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ace4ca-b41d-4c31-a3b1-f0f841b852c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:35.649024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ace4ca-b41d-4c31-a3b1-f0f841b852c0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:35.649307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ace4ca-b41d-4c31-a3b1-f0f841b852c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:35.649597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ace4ca-b41d-4c31-a3b1-f0f841b852c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:35.650326 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ace4ca-b41d-4c31-a3b1-f0f841b852c0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:35.650630 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ace4ca-b41d-4c31-a3b1-f0f841b852c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:35.652032 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ace4ca-b41d-4c31-a3b1-f0f841b852c0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:35.664407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 86/166] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:14:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:35.665218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ace4ca-b41d-4c31-a3b1-f0f841b852c0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:35.743095 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b0953-be32-466b-b3bb-9f4da7db687f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:35.743619 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b0953-be32-466b-b3bb-9f4da7db687f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:35.743904 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b0953-be32-466b-b3bb-9f4da7db687f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:35.744172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5b0953-be32-466b-b3bb-9f4da7db687f None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:35.744914 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b0953-be32-466b-b3bb-9f4da7db687f None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:35.745222 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b0953-be32-466b-b3bb-9f4da7db687f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:35.748300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5b0953-be32-466b-b3bb-9f4da7db687f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:35.771706 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 81/167] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 494 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:35.772475 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b0953-be32-466b-b3bb-9f4da7db687f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:35.878715 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:35.935326 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:35.936918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ace4ca-b41d-4c31-a3b1-f0f841b852c0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:35.979890 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5b0953-be32-466b-b3bb-9f4da7db687f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:36.012393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743dc52-012a-417c-a66a-0574ed95ad94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:36.012945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743dc52-012a-417c-a66a-0574ed95ad94 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:36.013232 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743dc52-012a-417c-a66a-0574ed95ad94 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:36.013513 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743dc52-012a-417c-a66a-0574ed95ad94 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:36.014273 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743dc52-012a-417c-a66a-0574ed95ad94 None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:36.014581 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743dc52-012a-417c-a66a-0574ed95ad94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:36.016235 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743dc52-012a-417c-a66a-0574ed95ad94 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:36.033112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 87/168] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:36.033981 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743dc52-012a-417c-a66a-0574ed95ad94 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:36.062882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d38a3-264e-441f-87fb-e5da29a2da13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:36.063889 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d38a3-264e-441f-87fb-e5da29a2da13 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:36.064363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d38a3-264e-441f-87fb-e5da29a2da13 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:36.064828 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d38a3-264e-441f-87fb-e5da29a2da13 None admin] PATH_INFO: `/v3/users/f7fc4b88abc344d2b72a4653dc811819` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:36.068861 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d38a3-264e-441f-87fb-e5da29a2da13 None admin] RBAC: Authorizing `identity:get_user(user_id=f7fc4b88abc344d2b72a4653dc811819)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:36.070578 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d38a3-264e-441f-87fb-e5da29a2da13 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:36.072414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 82/169] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:35 2022] GET /identity/v3/users/f7fc4b88abc344d2b72a4653dc811819 => generated 275 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:36.077465 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736c96-22c5-47e8-804f-42dd7a6ba0ae None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:36.078038 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736c96-22c5-47e8-804f-42dd7a6ba0ae None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:36.078520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736c96-22c5-47e8-804f-42dd7a6ba0ae None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:36.080097 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 83/170] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:36.085077 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98736c96-22c5-47e8-804f-42dd7a6ba0ae None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:36.151610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743dc52-012a-417c-a66a-0574ed95ad94 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:36.193729 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb859b81-f7b7-48bf-9fb7-0a06348d7b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:36.195668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb859b81-f7b7-48bf-9fb7-0a06348d7b37 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:36.201845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb859b81-f7b7-48bf-9fb7-0a06348d7b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:36.202358 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb859b81-f7b7-48bf-9fb7-0a06348d7b37 None admin] PATH_INFO: `/v3/users/f7fc4b88abc344d2b72a4653dc811819` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:36.203916 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb859b81-f7b7-48bf-9fb7-0a06348d7b37 None admin] RBAC: Authorizing `identity:get_user(user_id=f7fc4b88abc344d2b72a4653dc811819)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:36.205603 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb859b81-f7b7-48bf-9fb7-0a06348d7b37 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:36.207471 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 88/171] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:36 2022] GET /identity/v3/users/f7fc4b88abc344d2b72a4653dc811819 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:36.211109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb859b81-f7b7-48bf-9fb7-0a06348d7b37 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:36.242671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:36.338878 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98736c96-22c5-47e8-804f-42dd7a6ba0ae None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:36.374261 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb859b81-f7b7-48bf-9fb7-0a06348d7b37 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:36.378319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e625f1a-ca73-484f-85ba-2eef99893041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:36.379270 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e625f1a-ca73-484f-85ba-2eef99893041 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:36.379693 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e625f1a-ca73-484f-85ba-2eef99893041 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:36.380107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e625f1a-ca73-484f-85ba-2eef99893041 None admin] PATH_INFO: `/v3/users/f7fc4b88abc344d2b72a4653dc811819` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:36.382044 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e625f1a-ca73-484f-85ba-2eef99893041 None admin] RBAC: Authorizing `identity:get_user(user_id=f7fc4b88abc344d2b72a4653dc811819)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:36.384480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e625f1a-ca73-484f-85ba-2eef99893041 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:36.393951 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 84/172] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:36 2022] GET /identity/v3/users/f7fc4b88abc344d2b72a4653dc811819 => generated 275 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:36.396100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55520d0-e077-493d-982f-0037534c021a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:36.396533 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55520d0-e077-493d-982f-0037534c021a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:36.396947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55520d0-e077-493d-982f-0037534c021a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:36.412039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6536d6-e40a-4fde-8877-9bc393b5b66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:36.412657 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6536d6-e40a-4fde-8877-9bc393b5b66d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:36.412939 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6536d6-e40a-4fde-8877-9bc393b5b66d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:36.413208 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6536d6-e40a-4fde-8877-9bc393b5b66d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:36.424356 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b55520d0-e077-493d-982f-0037534c021a None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:36.425776 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b55520d0-e077-493d-982f-0037534c021a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:36.438442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:36.497835 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 85/173] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:36.498654 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6536d6-e40a-4fde-8877-9bc393b5b66d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:36.498859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55520d0-e077-493d-982f-0037534c021a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:36.500510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6536d6-e40a-4fde-8877-9bc393b5b66d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:36.519254 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:36.606134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55520d0-e077-493d-982f-0037534c021a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:36.619412 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 89/174] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:14:36.621797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa84f1c-4ccf-4bd9-9436-2d87aa966473 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:36.622427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa84f1c-4ccf-4bd9-9436-2d87aa966473 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:36.622990 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa84f1c-4ccf-4bd9-9436-2d87aa966473 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:36.624658 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 90/175] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:36.625537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa84f1c-4ccf-4bd9-9436-2d87aa966473 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:36.647207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2f914-ce1e-495a-aaf8-2bbb40a3cfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:36.647834 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2f914-ce1e-495a-aaf8-2bbb40a3cfba None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:36.648118 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2f914-ce1e-495a-aaf8-2bbb40a3cfba None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:36.648392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a2f914-ce1e-495a-aaf8-2bbb40a3cfba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:36.665224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:36.724572 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2f914-ce1e-495a-aaf8-2bbb40a3cfba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:36.726100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a2f914-ce1e-495a-aaf8-2bbb40a3cfba None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:36.740916 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa84f1c-4ccf-4bd9-9436-2d87aa966473 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:36.756887 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:36.782703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3056095-f4e8-4296-ba86-d549bce2c1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:36.783326 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3056095-f4e8-4296-ba86-d549bce2c1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:36.783605 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3056095-f4e8-4296-ba86-d549bce2c1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:36.783877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3056095-f4e8-4296-ba86-d549bce2c1a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:36.836420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 86/176] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:14:36.837062 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2f914-ce1e-495a-aaf8-2bbb40a3cfba None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:36.858638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3056095-f4e8-4296-ba86-d549bce2c1a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:36.860042 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3056095-f4e8-4296-ba86-d549bce2c1a9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:36.884196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:36.939099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a2f914-ce1e-495a-aaf8-2bbb40a3cfba None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:36.966525 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 91/177] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:14:36.967692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6681e-5b63-42e9-84cc-7d39bdf353ff None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:36.967975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6681e-5b63-42e9-84cc-7d39bdf353ff None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:36.968244 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6681e-5b63-42e9-84cc-7d39bdf353ff None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:36.973418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 92/178] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:36.973418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6681e-5b63-42e9-84cc-7d39bdf353ff None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:36.981929 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cef5e-8731-4a15-9d20-eb65e9cf3132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:36.982580 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cef5e-8731-4a15-9d20-eb65e9cf3132 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:36.982860 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cef5e-8731-4a15-9d20-eb65e9cf3132 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:36.983131 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cef5e-8731-4a15-9d20-eb65e9cf3132 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:36.987726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:36.994469 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482cef5e-8731-4a15-9d20-eb65e9cf3132 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:36.999912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-482cef5e-8731-4a15-9d20-eb65e9cf3132 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:14:36.999912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:36.999912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:36.999912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:36.999912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:36.999912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:36.999912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:36.999912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:36.999912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:36.999912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:36.999912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:36.999912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:36.999912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:36.999912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:36.999912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:36.999912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:37.000809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:37.000809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:37.000809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:37.000809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:37.000809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:37.000809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:37.000809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:37.000809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:37.000809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:37.000809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:37.000809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:37.000809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:37.000809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:37.000809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:37.000809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:37.000809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:37.001630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:37.001630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:37.001630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:37.001630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:37.001630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:37.001630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:37.001630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:37.001630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:37.001630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:37.001630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:37.001630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:14:37.001630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:37.001630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 87/179] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:14:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:37.002319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdbba58-d7f7-4a9d-993d-04541ea96c13 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:37.002722 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdbba58-d7f7-4a9d-993d-04541ea96c13 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:37.002850 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdbba58-d7f7-4a9d-993d-04541ea96c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:37.021643 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3bdbba58-d7f7-4a9d-993d-04541ea96c13 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:37.022824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3bdbba58-d7f7-4a9d-993d-04541ea96c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:37.091529 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6681e-5b63-42e9-84cc-7d39bdf353ff None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:37.094820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 88/180] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:37.102870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdbba58-d7f7-4a9d-993d-04541ea96c13 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:37.129944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800297bb-4d06-4425-8314-4e9afbb69855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:37.130507 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800297bb-4d06-4425-8314-4e9afbb69855 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:37.130813 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800297bb-4d06-4425-8314-4e9afbb69855 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:37.131096 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800297bb-4d06-4425-8314-4e9afbb69855 None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:37.132285 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800297bb-4d06-4425-8314-4e9afbb69855 None admin] RBAC: Authorizing `identity:get_project(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:37.133752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800297bb-4d06-4425-8314-4e9afbb69855 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:37.135543 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 93/181] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:36 2022] GET /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:37.136155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800297bb-4d06-4425-8314-4e9afbb69855 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:37.147967 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:37.217019 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdbba58-d7f7-4a9d-993d-04541ea96c13 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:37.219898 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:37.239378 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800297bb-4d06-4425-8314-4e9afbb69855 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:37.249839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd140c-233a-45b0-80b5-62105f470df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:37.250479 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd140c-233a-45b0-80b5-62105f470df1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:37.250757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd140c-233a-45b0-80b5-62105f470df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:37.251025 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd140c-233a-45b0-80b5-62105f470df1 None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:37.252165 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd140c-233a-45b0-80b5-62105f470df1 None admin] RBAC: Authorizing `identity:get_project(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:37.253628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fd140c-233a-45b0-80b5-62105f470df1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:37.255284 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 89/182] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:37 2022] GET /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:37.256374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daf060-afe3-478b-8e42-2882dd14d13c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:37.256652 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daf060-afe3-478b-8e42-2882dd14d13c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:37.256922 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03daf060-afe3-478b-8e42-2882dd14d13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:37.274533 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-03daf060-afe3-478b-8e42-2882dd14d13c None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:37.275703 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-03daf060-afe3-478b-8e42-2882dd14d13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:37.278724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c323cd-d218-4c6d-a4a5-759d11fa9b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:37.279372 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c323cd-d218-4c6d-a4a5-759d11fa9b62 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:37.279650 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c323cd-d218-4c6d-a4a5-759d11fa9b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:37.279921 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c323cd-d218-4c6d-a4a5-759d11fa9b62 None admin] PATH_INFO: `/v3/projects/cee19bd514184a9988cbe093840d4283` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:37.281093 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c323cd-d218-4c6d-a4a5-759d11fa9b62 None admin] RBAC: Authorizing `identity:get_project(project_id=cee19bd514184a9988cbe093840d4283)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:37.282594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c323cd-d218-4c6d-a4a5-759d11fa9b62 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:37.284270 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 94/183] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:37 2022] GET /identity/v3/projects/cee19bd514184a9988cbe093840d4283 => generated 312 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:37.285021 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c323cd-d218-4c6d-a4a5-759d11fa9b62 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:37.342835 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 90/184] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:37.343717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03daf060-afe3-478b-8e42-2882dd14d13c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:37.377600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:37.386586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c323cd-d218-4c6d-a4a5-759d11fa9b62 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:37.420300 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f094c6d-e47b-4e4f-a5b9-dc1c9fee5291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:37.420914 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f094c6d-e47b-4e4f-a5b9-dc1c9fee5291 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:37.421192 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f094c6d-e47b-4e4f-a5b9-dc1c9fee5291 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:37.421461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f094c6d-e47b-4e4f-a5b9-dc1c9fee5291 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:37.422213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f094c6d-e47b-4e4f-a5b9-dc1c9fee5291 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:37.422510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f094c6d-e47b-4e4f-a5b9-dc1c9fee5291 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:37.423889 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f094c6d-e47b-4e4f-a5b9-dc1c9fee5291 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:37.431835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 95/185] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:37.434967 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff762f97-719a-44cc-bbd3-ba34bc51dc6b None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:37.435278 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff762f97-719a-44cc-bbd3-ba34bc51dc6b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:37.435556 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff762f97-719a-44cc-bbd3-ba34bc51dc6b None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:37.436888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 96/186] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:37.437615 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff762f97-719a-44cc-bbd3-ba34bc51dc6b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:37.447070 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03daf060-afe3-478b-8e42-2882dd14d13c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:37.449985 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:37.481695 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389ee6c-3d7b-4e18-84bb-7cc4255a3abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:37.482332 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389ee6c-3d7b-4e18-84bb-7cc4255a3abc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:37.482610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389ee6c-3d7b-4e18-84bb-7cc4255a3abc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:37.482879 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389ee6c-3d7b-4e18-84bb-7cc4255a3abc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:37.484004 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389ee6c-3d7b-4e18-84bb-7cc4255a3abc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:37.485439 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389ee6c-3d7b-4e18-84bb-7cc4255a3abc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:37.487086 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 91/187] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:14:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:37.489778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389ee6c-3d7b-4e18-84bb-7cc4255a3abc None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:37.541535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff762f97-719a-44cc-bbd3-ba34bc51dc6b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:37.576374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b42f1-137e-4439-a12a-61074b86fa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:37.576992 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b42f1-137e-4439-a12a-61074b86fa37 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:37.577276 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b42f1-137e-4439-a12a-61074b86fa37 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:37.577611 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405b42f1-137e-4439-a12a-61074b86fa37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:37.578410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405b42f1-137e-4439-a12a-61074b86fa37 None admin] RBAC: Adding query filter params (role.id=4146966c23674402aebb9a5b21cf18ba, scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29, user.id=f7fc4b88abc344d2b72a4653dc811819) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:37.578725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405b42f1-137e-4439-a12a-61074b86fa37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:37.580192 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405b42f1-137e-4439-a12a-61074b86fa37 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:37.585852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 97/188] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:14:37 2022] GET /identity/v3/role_assignments?user.id=f7fc4b88abc344d2b72a4653dc811819&scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29&role.id=4146966c23674402aebb9a5b21cf18ba => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:37.586629 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b42f1-137e-4439-a12a-61074b86fa37 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:37.586806 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389ee6c-3d7b-4e18-84bb-7cc4255a3abc None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:37.588290 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:37.627037 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a674dc-3fe1-4b6b-a253-3d53c7efdb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:37.627671 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a674dc-3fe1-4b6b-a253-3d53c7efdb94 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:37.627954 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a674dc-3fe1-4b6b-a253-3d53c7efdb94 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:37.628229 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a674dc-3fe1-4b6b-a253-3d53c7efdb94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:37.629041 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a674dc-3fe1-4b6b-a253-3d53c7efdb94 None admin] RBAC: Adding query filter params (role.id=4912a35790fe4d4297540156083d81d1, scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29, user.id=f7fc4b88abc344d2b72a4653dc811819) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:37.629351 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a674dc-3fe1-4b6b-a253-3d53c7efdb94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:37.630861 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a674dc-3fe1-4b6b-a253-3d53c7efdb94 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:37.638456 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 92/189] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:14:37 2022] GET /identity/v3/role_assignments?user.id=f7fc4b88abc344d2b72a4653dc811819&scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29&role.id=4912a35790fe4d4297540156083d81d1 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:37.639260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a674dc-3fe1-4b6b-a253-3d53c7efdb94 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:37.726096 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:37.767956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405b42f1-137e-4439-a12a-61074b86fa37 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:37.800620 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a674dc-3fe1-4b6b-a253-3d53c7efdb94 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:37.809341 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ba6d6-2c9f-403c-adde-48c6cfe7f51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:37.810118 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ba6d6-2c9f-403c-adde-48c6cfe7f51d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:37.810894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ba6d6-2c9f-403c-adde-48c6cfe7f51d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:37.810894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ba6d6-2c9f-403c-adde-48c6cfe7f51d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:37.811519 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ba6d6-2c9f-403c-adde-48c6cfe7f51d None admin] RBAC: Adding query filter params (role.id=4912a35790fe4d4297540156083d81d1, scope.project.id=cee19bd514184a9988cbe093840d4283, user.id=f7fc4b88abc344d2b72a4653dc811819) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:37.811840 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ba6d6-2c9f-403c-adde-48c6cfe7f51d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:37.813301 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5ba6d6-2c9f-403c-adde-48c6cfe7f51d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:37.822032 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 98/190] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:14:37 2022] GET /identity/v3/role_assignments?user.id=f7fc4b88abc344d2b72a4653dc811819&scope.project.id=cee19bd514184a9988cbe093840d4283&role.id=4912a35790fe4d4297540156083d81d1 => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:37.822867 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ba6d6-2c9f-403c-adde-48c6cfe7f51d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:37.843286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3919f7f8-1246-44a5-84d6-f5795505d6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:37.843951 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919f7f8-1246-44a5-84d6-f5795505d6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:37.844229 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919f7f8-1246-44a5-84d6-f5795505d6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:37.844497 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3919f7f8-1246-44a5-84d6-f5795505d6f2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:37.868531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3919f7f8-1246-44a5-84d6-f5795505d6f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:37.875855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-3919f7f8-1246-44a5-84d6-f5795505d6f2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:14:37.875855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:37.875855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:37.875855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:37.875855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:37.875855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:37.875855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:37.875855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:37.875855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:37.875855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:37.875855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:37.875855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:37.875855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:37.875855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:37.875855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:37.875855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:37.876789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:37.876789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:37.876789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:37.876789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:37.876789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:37.876789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:37.876789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:37.876789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:37.876789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:37.876789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:37.876789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:37.876789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:37.876789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:37.876789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:37.876789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:37.876789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:37.877784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:37.877784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:37.877784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:37.877784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:37.877784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:37.877784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:37.877784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:37.877784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:37.877784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:37.877784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:37.877784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:14:37.877784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:37.877784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 93/191] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:14:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:37.878572 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94903f6a-8507-4497-b473-025f4561e4f0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:37.878692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94903f6a-8507-4497-b473-025f4561e4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:37.878962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94903f6a-8507-4497-b473-025f4561e4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:37.903504 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-94903f6a-8507-4497-b473-025f4561e4f0 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:37.904682 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-94903f6a-8507-4497-b473-025f4561e4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:37.930172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:37.964023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ba6d6-2c9f-403c-adde-48c6cfe7f51d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:37.998296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 94/192] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:37.999227 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94903f6a-8507-4497-b473-025f4561e4f0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:38.008526 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597c7ee-b236-495b-a4e9-1e25e353c256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:38.009183 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597c7ee-b236-495b-a4e9-1e25e353c256 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:38.009463 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597c7ee-b236-495b-a4e9-1e25e353c256 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:38.009770 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c597c7ee-b236-495b-a4e9-1e25e353c256 None admin] PATH_INFO: `/v3/users/157e0b33fb04490fbea290fbd4683c3f` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:38.021687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:38.032212 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597c7ee-b236-495b-a4e9-1e25e353c256 None admin] RBAC: Authorizing `identity:get_user(user_id=157e0b33fb04490fbea290fbd4683c3f)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:38.033632 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c597c7ee-b236-495b-a4e9-1e25e353c256 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:38.034926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 99/193] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:37 2022] GET /identity/v3/users/157e0b33fb04490fbea290fbd4683c3f => generated 283 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:38.035530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597c7ee-b236-495b-a4e9-1e25e353c256 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:38.123597 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94903f6a-8507-4497-b473-025f4561e4f0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:38.148820 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c597c7ee-b236-495b-a4e9-1e25e353c256 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:38.160783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340166d2-c47e-4204-870a-e3d27964dd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:38.161414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340166d2-c47e-4204-870a-e3d27964dd08 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:38.161726 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340166d2-c47e-4204-870a-e3d27964dd08 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:38.162003 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340166d2-c47e-4204-870a-e3d27964dd08 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:38.162686 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340166d2-c47e-4204-870a-e3d27964dd08 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:38.164134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340166d2-c47e-4204-870a-e3d27964dd08 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:38.184350 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 95/194] 10.222.0.39 () {60 vars in 1132 bytes} [Thu Jul 28 22:14:37 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:14:38.185172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340166d2-c47e-4204-870a-e3d27964dd08 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:38.187178 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7989f-ddf9-4162-89b5-b374bfbc4ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:38.187931 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7989f-ddf9-4162-89b5-b374bfbc4ecd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:38.188207 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7989f-ddf9-4162-89b5-b374bfbc4ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:38.188619 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7989f-ddf9-4162-89b5-b374bfbc4ecd None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:38.189352 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7989f-ddf9-4162-89b5-b374bfbc4ecd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:38.189722 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7989f-ddf9-4162-89b5-b374bfbc4ecd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:38.191110 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd7989f-ddf9-4162-89b5-b374bfbc4ecd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:38.201320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:38.210344 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 100/195] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:14:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:38.210344 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7989f-ddf9-4162-89b5-b374bfbc4ecd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:38.274513 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:38.307797 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340166d2-c47e-4204-870a-e3d27964dd08 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:38.324173 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7989f-ddf9-4162-89b5-b374bfbc4ecd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:38.348203 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938740ef-0983-4a44-9898-8919c2648347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:38.348945 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938740ef-0983-4a44-9898-8919c2648347 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:38.349293 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938740ef-0983-4a44-9898-8919c2648347 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:38.349662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938740ef-0983-4a44-9898-8919c2648347 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:38.365505 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938740ef-0983-4a44-9898-8919c2648347 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:38.371426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-938740ef-0983-4a44-9898-8919c2648347 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 22:14:38.371426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:38.371426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:38.371426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:38.371426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:38.371426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:38.371426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:38.371426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:38.371426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:38.371426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:38.371426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:38.371426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:38.371426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:38.371426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:38.371426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:38.371426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:38.372325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:38.372325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:38.372325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:38.372325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:38.372325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:38.372325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:38.372325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:38.372325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:38.372325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:38.372325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:38.372325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:38.372325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:38.372325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:38.372325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:38.372325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:38.372325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:38.373100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:38.373100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:38.373100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:38.373100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:38.373100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:38.373100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:38.373100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:38.373100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:38.373100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:38.373100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:38.373100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 22:14:38.373100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:38.373100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 96/196] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:38.373827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938740ef-0983-4a44-9898-8919c2648347 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:38.379864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb11a7-71d6-40fd-bc96-1c183b875548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:38.380529 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb11a7-71d6-40fd-bc96-1c183b875548 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:38.380815 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb11a7-71d6-40fd-bc96-1c183b875548 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:38.381090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb11a7-71d6-40fd-bc96-1c183b875548 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:38.478544 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb11a7-71d6-40fd-bc96-1c183b875548 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:38.479947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:38.481993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb11a7-71d6-40fd-bc96-1c183b875548 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:38.519793 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938740ef-0983-4a44-9898-8919c2648347 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:38.553433 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:38.571098 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bfc2e1-6433-4485-9bbb-4409814b3bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:38.571749 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfc2e1-6433-4485-9bbb-4409814b3bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:38.572031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfc2e1-6433-4485-9bbb-4409814b3bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:38.572301 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bfc2e1-6433-4485-9bbb-4409814b3bf4 None admin] PATH_INFO: `/v3/users/157e0b33fb04490fbea290fbd4683c3f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:38.574688 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bfc2e1-6433-4485-9bbb-4409814b3bf4 None admin] RBAC: Authorizing `identity:get_user(user_id=157e0b33fb04490fbea290fbd4683c3f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:38.576765 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bfc2e1-6433-4485-9bbb-4409814b3bf4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:38.579132 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 97/197] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:38 2022] GET /identity/v3/users/157e0b33fb04490fbea290fbd4683c3f => generated 283 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:38.580155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bfc2e1-6433-4485-9bbb-4409814b3bf4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:38.606579 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 101/198] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:14:38.607435 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb11a7-71d6-40fd-bc96-1c183b875548 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:38.718384 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bfc2e1-6433-4485-9bbb-4409814b3bf4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:38.722965 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb11a7-71d6-40fd-bc96-1c183b875548 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:38.762349 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b00a6-8570-4db5-97f3-cdefc50ec457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:38.763364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b00a6-8570-4db5-97f3-cdefc50ec457 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:38.763646 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b00a6-8570-4db5-97f3-cdefc50ec457 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:38.763914 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b00a6-8570-4db5-97f3-cdefc50ec457 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:38.765374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b00a6-8570-4db5-97f3-cdefc50ec457 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:38.765715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b00a6-8570-4db5-97f3-cdefc50ec457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:38.767395 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b00a6-8570-4db5-97f3-cdefc50ec457 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:38.776091 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb755917-7350-4173-9606-20da12a2f4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:38.776993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb755917-7350-4173-9606-20da12a2f4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:38.777343 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb755917-7350-4173-9606-20da12a2f4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:38.777649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb755917-7350-4173-9606-20da12a2f4d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:38.779198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 98/199] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:38.780107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b00a6-8570-4db5-97f3-cdefc50ec457 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:38.783072 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:38.842471 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:38.873378 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb755917-7350-4173-9606-20da12a2f4d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:38.875358 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb755917-7350-4173-9606-20da12a2f4d9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:38.903608 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b00a6-8570-4db5-97f3-cdefc50ec457 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:38.946106 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c261a5-27f7-4cf2-a4b7-00f19ebcf1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:38.947161 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c261a5-27f7-4cf2-a4b7-00f19ebcf1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:38.947713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c261a5-27f7-4cf2-a4b7-00f19ebcf1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:38.947994 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c261a5-27f7-4cf2-a4b7-00f19ebcf1e4 None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:38.949740 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c261a5-27f7-4cf2-a4b7-00f19ebcf1e4 None admin] RBAC: Authorizing `identity:get_project(project_id=e5b44df13345468b90906e747c09a977)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:38.951936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c261a5-27f7-4cf2-a4b7-00f19ebcf1e4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:38.954132 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 99/200] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:38 2022] GET /identity/v3/projects/e5b44df13345468b90906e747c09a977 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:38.955154 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c261a5-27f7-4cf2-a4b7-00f19ebcf1e4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:39.009008 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 102/201] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:14:39.013989 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb755917-7350-4173-9606-20da12a2f4d9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:39.050868 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:39.086562 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c261a5-27f7-4cf2-a4b7-00f19ebcf1e4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:39.095906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:39.122600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116d320-353d-4e95-9b57-37ae6135d1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:39.123412 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116d320-353d-4e95-9b57-37ae6135d1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:39.123820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116d320-353d-4e95-9b57-37ae6135d1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:39.124189 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116d320-353d-4e95-9b57-37ae6135d1d4 None admin] PATH_INFO: `/v3/users/ce54aa78635b42ed9decf8c08ae6465f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:39.125476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116d320-353d-4e95-9b57-37ae6135d1d4 None admin] RBAC: Authorizing `identity:get_user(user_id=ce54aa78635b42ed9decf8c08ae6465f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:39.127063 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5116d320-353d-4e95-9b57-37ae6135d1d4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:39.129053 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 100/202] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:38 2022] GET /identity/v3/users/ce54aa78635b42ed9decf8c08ae6465f => generated 289 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:39.129951 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116d320-353d-4e95-9b57-37ae6135d1d4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:39.149356 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb755917-7350-4173-9606-20da12a2f4d9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:39.188808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb4ff4-65c7-4609-9972-faf1c175852a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:39.189431 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb4ff4-65c7-4609-9972-faf1c175852a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:39.189746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb4ff4-65c7-4609-9972-faf1c175852a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:39.190032 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb4ff4-65c7-4609-9972-faf1c175852a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:39.190934 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb4ff4-65c7-4609-9972-faf1c175852a None admin] RBAC: Adding query filter params (role.id=aded7bd43e974686b3c23bd9d4255b24, scope.project.id=e5b44df13345468b90906e747c09a977, user.id=157e0b33fb04490fbea290fbd4683c3f) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:39.191360 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb4ff4-65c7-4609-9972-faf1c175852a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:39.192822 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb4ff4-65c7-4609-9972-faf1c175852a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:39.199910 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 103/203] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:14:39 2022] GET /identity/v3/role_assignments?user.id=157e0b33fb04490fbea290fbd4683c3f&scope.project.id=e5b44df13345468b90906e747c09a977&role.id=aded7bd43e974686b3c23bd9d4255b24 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:39.200817 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb4ff4-65c7-4609-9972-faf1c175852a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:39.296647 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116d320-353d-4e95-9b57-37ae6135d1d4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:39.318186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb4ff4-65c7-4609-9972-faf1c175852a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:39.344022 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:39.358115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9331523-5a0d-4b8f-b88b-be119125cd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:39.358887 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9331523-5a0d-4b8f-b88b-be119125cd31 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:39.359172 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9331523-5a0d-4b8f-b88b-be119125cd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:39.359452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9331523-5a0d-4b8f-b88b-be119125cd31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:39.397080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09550a82-cc57-4c90-9932-e47bcef2af2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:39.397774 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09550a82-cc57-4c90-9932-e47bcef2af2f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:39.398053 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09550a82-cc57-4c90-9932-e47bcef2af2f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:39.398324 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09550a82-cc57-4c90-9932-e47bcef2af2f None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:39.399506 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09550a82-cc57-4c90-9932-e47bcef2af2f None admin] RBAC: Authorizing `identity:get_project(project_id=e5b44df13345468b90906e747c09a977)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:39.400961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09550a82-cc57-4c90-9932-e47bcef2af2f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:39.403125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 101/204] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:39 2022] GET /identity/v3/projects/e5b44df13345468b90906e747c09a977 => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:39.408626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09550a82-cc57-4c90-9932-e47bcef2af2f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:39.419819 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:39.438436 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9331523-5a0d-4b8f-b88b-be119125cd31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:39.439942 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9331523-5a0d-4b8f-b88b-be119125cd31 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:39.552332 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 104/205] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:14:39.557866 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9331523-5a0d-4b8f-b88b-be119125cd31 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:39.580756 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09550a82-cc57-4c90-9932-e47bcef2af2f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:39.586113 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:39.664642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9331523-5a0d-4b8f-b88b-be119125cd31 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:39.701999 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db540f-f955-4be8-aa0e-fcffde85820f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:39.702651 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db540f-f955-4be8-aa0e-fcffde85820f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:39.702937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db540f-f955-4be8-aa0e-fcffde85820f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:39.703214 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db540f-f955-4be8-aa0e-fcffde85820f None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:39.704400 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db540f-f955-4be8-aa0e-fcffde85820f None admin] RBAC: Authorizing `identity:get_project(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:39.705894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9db540f-f955-4be8-aa0e-fcffde85820f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:39.712482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 105/206] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:39 2022] GET /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:39.713680 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db540f-f955-4be8-aa0e-fcffde85820f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:39.718245 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1d6912-8748-4778-8841-e96db03fb3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:39.718893 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d6912-8748-4778-8841-e96db03fb3eb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:39.719176 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d6912-8748-4778-8841-e96db03fb3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:39.719454 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1d6912-8748-4778-8841-e96db03fb3eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:39.720255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d6912-8748-4778-8841-e96db03fb3eb None admin] RBAC: Adding query filter params (role.id=4146966c23674402aebb9a5b21cf18ba, scope.project.id=e5b44df13345468b90906e747c09a977, user.id=157e0b33fb04490fbea290fbd4683c3f) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:39.720565 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d6912-8748-4778-8841-e96db03fb3eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:39.734505 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1d6912-8748-4778-8841-e96db03fb3eb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:39.744575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 102/207] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:14:39 2022] GET /identity/v3/role_assignments?user.id=157e0b33fb04490fbea290fbd4683c3f&scope.project.id=e5b44df13345468b90906e747c09a977&role.id=4146966c23674402aebb9a5b21cf18ba => generated 259 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:39.825865 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db540f-f955-4be8-aa0e-fcffde85820f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:39.834167 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:39.867517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289fee0b-4474-40c9-9c54-78311f987259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:39.868152 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fee0b-4474-40c9-9c54-78311f987259 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:39.868444 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fee0b-4474-40c9-9c54-78311f987259 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:39.868721 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289fee0b-4474-40c9-9c54-78311f987259 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:39.869559 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289fee0b-4474-40c9-9c54-78311f987259 None admin] RBAC: Adding query filter params (role.id=d2fa7787cf3f461589be9f37afe9de6e, scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29, user.id=ce54aa78635b42ed9decf8c08ae6465f) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:39.869887 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289fee0b-4474-40c9-9c54-78311f987259 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:39.872051 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289fee0b-4474-40c9-9c54-78311f987259 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:39.884345 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 106/208] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:14:39 2022] GET /identity/v3/role_assignments?user.id=ce54aa78635b42ed9decf8c08ae6465f&scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29&role.id=d2fa7787cf3f461589be9f37afe9de6e => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:40.487619 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1570a51-4a43-4d2c-b6dd-37870608aba0 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:40.488283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1570a51-4a43-4d2c-b6dd-37870608aba0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:40.488283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1570a51-4a43-4d2c-b6dd-37870608aba0 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:40.489720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 103/209] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:40.495305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b51e7-525e-4472-8937-0d973adfe714 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:40.495845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b51e7-525e-4472-8937-0d973adfe714 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:40.496640 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b51e7-525e-4472-8937-0d973adfe714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:40.516608 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6c7b51e7-525e-4472-8937-0d973adfe714 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:40.517845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6c7b51e7-525e-4472-8937-0d973adfe714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:40.591123 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 107/210] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:40.607616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5a0e4-d3db-4d93-9c14-d2bb8ff7b05c None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:40.607907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5a0e4-d3db-4d93-9c14-d2bb8ff7b05c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:40.608187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f5a0e4-d3db-4d93-9c14-d2bb8ff7b05c None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:40.609719 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 104/211] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:40.615161 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62c651-02f4-4553-b543-9c7d28453fcd None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:40.615509 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62c651-02f4-4553-b543-9c7d28453fcd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:40.615791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e62c651-02f4-4553-b543-9c7d28453fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:40.617460 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f5a0e4-d3db-4d93-9c14-d2bb8ff7b05c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:40.642805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3e62c651-02f4-4553-b543-9c7d28453fcd None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:40.644616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3e62c651-02f4-4553-b543-9c7d28453fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:40.702194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:40.740968 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 108/212] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:40.753709 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f5a0e4-d3db-4d93-9c14-d2bb8ff7b05c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:40.765488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62c651-02f4-4553-b543-9c7d28453fcd None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:40.783827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:40.802310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd1947-18e3-489c-99bd-25dcd1c4cbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:40.803370 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd1947-18e3-489c-99bd-25dcd1c4cbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:40.803650 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd1947-18e3-489c-99bd-25dcd1c4cbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:40.804187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd1947-18e3-489c-99bd-25dcd1c4cbf3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:40.818610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd1947-18e3-489c-99bd-25dcd1c4cbf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:40.826524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7bbd1947-18e3-489c-99bd-25dcd1c4cbf3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:40.826524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:40.826524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:40.826524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:40.826524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:40.826524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:40.826524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:40.826524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:40.826524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:40.826524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:40.826524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:40.826524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:40.826524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:40.826524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:40.826524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:40.826524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:40.827390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:40.827390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:40.827390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:40.827390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:40.827390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:40.827390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:40.827390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:40.827390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:40.827390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:40.827390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:40.827390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:40.827390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:40.827390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:40.827390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:40.827390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:40.827390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:40.828172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:40.828172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:40.828172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:40.828172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:40.828172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:40.828172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:40.828172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:40.828172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:40.828172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:40.828172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:40.828172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:40.828172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:40.828172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 105/213] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:14:40.835095 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd1947-18e3-489c-99bd-25dcd1c4cbf3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:40.881965 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e62c651-02f4-4553-b543-9c7d28453fcd None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:40.916901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124e4fa-cf86-4739-9767-c12e2d1e7b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:40.918278 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124e4fa-cf86-4739-9767-c12e2d1e7b4d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:40.918435 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124e4fa-cf86-4739-9767-c12e2d1e7b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:40.918658 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124e4fa-cf86-4739-9767-c12e2d1e7b4d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:40.919778 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124e4fa-cf86-4739-9767-c12e2d1e7b4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:40.921245 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2124e4fa-cf86-4739-9767-c12e2d1e7b4d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:40.922885 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 109/214] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:14:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:40.928265 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124e4fa-cf86-4739-9767-c12e2d1e7b4d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:40.967608 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd1947-18e3-489c-99bd-25dcd1c4cbf3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:41.003275 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:41.010522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007148f-5c58-410f-959c-0cbdbf59b5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:41.012235 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007148f-5c58-410f-959c-0cbdbf59b5af None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:41.012646 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007148f-5c58-410f-959c-0cbdbf59b5af None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:41.013219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007148f-5c58-410f-959c-0cbdbf59b5af None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:41.014297 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007148f-5c58-410f-959c-0cbdbf59b5af None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:41.014713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007148f-5c58-410f-959c-0cbdbf59b5af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:41.016637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007148f-5c58-410f-959c-0cbdbf59b5af None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:41.022112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:41.028640 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 106/215] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:41.034625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007148f-5c58-410f-959c-0cbdbf59b5af None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:41.044503 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2124e4fa-cf86-4739-9767-c12e2d1e7b4d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:41.082570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b330aa-ff3b-4d89-a909-619647bd9853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:41.084210 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b330aa-ff3b-4d89-a909-619647bd9853 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:41.084568 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b330aa-ff3b-4d89-a909-619647bd9853 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:41.084916 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b330aa-ff3b-4d89-a909-619647bd9853 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:41.085744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b330aa-ff3b-4d89-a909-619647bd9853 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:41.087274 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b330aa-ff3b-4d89-a909-619647bd9853 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:41.121630 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 110/216] 10.222.0.39 () {60 vars in 1132 bytes} [Thu Jul 28 22:14:40 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:14:41.184908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007148f-5c58-410f-959c-0cbdbf59b5af None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:41.225968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f0ffd-edad-49bf-bcc3-fabac24c2966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:41.226689 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f0ffd-edad-49bf-bcc3-fabac24c2966 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:41.226968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f0ffd-edad-49bf-bcc3-fabac24c2966 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:41.227249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f0ffd-edad-49bf-bcc3-fabac24c2966 None admin] PATH_INFO: `/v3/users/6e1b83f0ecf54ae88fc0f287046ea728` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:41.243153 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f0ffd-edad-49bf-bcc3-fabac24c2966 None admin] RBAC: Authorizing `identity:get_user(user_id=6e1b83f0ecf54ae88fc0f287046ea728)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:41.244729 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f0ffd-edad-49bf-bcc3-fabac24c2966 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:41.247569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 107/217] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:41 2022] GET /identity/v3/users/6e1b83f0ecf54ae88fc0f287046ea728 => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:41.254839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b330aa-ff3b-4d89-a909-619647bd9853 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:41.363269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b330aa-ff3b-4d89-a909-619647bd9853 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:41.399161 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69acc7b-5fa0-4984-9f80-e9c2936db241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:41.399780 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69acc7b-5fa0-4984-9f80-e9c2936db241 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:41.400065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69acc7b-5fa0-4984-9f80-e9c2936db241 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:41.400339 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69acc7b-5fa0-4984-9f80-e9c2936db241 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:41.410001 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:41.471969 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69acc7b-5fa0-4984-9f80-e9c2936db241 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:41.473455 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69acc7b-5fa0-4984-9f80-e9c2936db241 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:41.578836 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 111/218] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:14:41.585038 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f0ffd-edad-49bf-bcc3-fabac24c2966 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:41.635154 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95095d4-9610-4dac-8060-9e2c0653f69b None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:41.635475 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95095d4-9610-4dac-8060-9e2c0653f69b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:41.635707 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95095d4-9610-4dac-8060-9e2c0653f69b None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:41.637121 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 112/219] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:14:41.642675 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfeb2d4-485a-46be-87d7-fe1600050f56 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:41.643188 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfeb2d4-485a-46be-87d7-fe1600050f56 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:41.643681 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfeb2d4-485a-46be-87d7-fe1600050f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:41.649021 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:41.670154 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6bfeb2d4-485a-46be-87d7-fe1600050f56 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:41.671878 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6bfeb2d4-485a-46be-87d7-fe1600050f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:41.733920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:41.773964 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f0ffd-edad-49bf-bcc3-fabac24c2966 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:41.798679 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 113/220] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:14:41.821132 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfeb2d4-485a-46be-87d7-fe1600050f56 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:41.842281 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01470486-3290-42e6-b535-6510541a6c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:41.842859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01470486-3290-42e6-b535-6510541a6c60 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:41.843143 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01470486-3290-42e6-b535-6510541a6c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:41.843417 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01470486-3290-42e6-b535-6510541a6c60 None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:41.848807 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01470486-3290-42e6-b535-6510541a6c60 None admin] RBAC: Authorizing `identity:get_project(project_id=e5b44df13345468b90906e747c09a977)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:41.850349 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01470486-3290-42e6-b535-6510541a6c60 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:41.852255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 108/221] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:41 2022] GET /identity/v3/projects/e5b44df13345468b90906e747c09a977 => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:41.859381 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01470486-3290-42e6-b535-6510541a6c60 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:42.002881 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfeb2d4-485a-46be-87d7-fe1600050f56 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:42.053556 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01470486-3290-42e6-b535-6510541a6c60 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:42.055688 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16c9a7-bf77-4bad-b705-b3978b1c9496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:42.057218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16c9a7-bf77-4bad-b705-b3978b1c9496 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:42.057507 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16c9a7-bf77-4bad-b705-b3978b1c9496 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:42.057817 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16c9a7-bf77-4bad-b705-b3978b1c9496 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:42.071995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d16c9a7-bf77-4bad-b705-b3978b1c9496 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:42.073329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:42.079757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-8d16c9a7-bf77-4bad-b705-b3978b1c9496 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:14:42.079757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:42.079757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:42.079757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:42.079757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:42.079757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:42.079757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:42.079757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:42.079757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:42.079757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:42.079757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:42.079757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:42.079757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:42.079757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:42.079757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:42.079757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:42.080674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:42.080674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:42.080674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:42.080674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:42.080674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:42.080674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:42.080674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:42.080674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:42.080674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:42.080674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:42.080674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:42.080674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:42.080674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:42.080674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:42.080674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:42.080674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:42.081467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:42.081467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:42.081467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:42.081467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:42.081467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:42.081467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:42.081467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:42.081467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:42.081467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:42.081467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:42.081467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:14:42.081467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:14:42.082573 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 114/222] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:14:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:14:42.084350 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2226b-78c2-46cd-9b5b-7670b8f2857b None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:42.084642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2226b-78c2-46cd-9b5b-7670b8f2857b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:42.084927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b2226b-78c2-46cd-9b5b-7670b8f2857b None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:42.086599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 115/223] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:42.087452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b2226b-78c2-46cd-9b5b-7670b8f2857b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:42.105803 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c911c49-f9d2-4cd8-8fc9-69cecf7bf32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:42.106621 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c911c49-f9d2-4cd8-8fc9-69cecf7bf32c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:42.106918 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c911c49-f9d2-4cd8-8fc9-69cecf7bf32c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:42.107195 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c911c49-f9d2-4cd8-8fc9-69cecf7bf32c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:42.108116 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c911c49-f9d2-4cd8-8fc9-69cecf7bf32c None admin] RBAC: Adding query filter params (role.id=4912a35790fe4d4297540156083d81d1, scope.project.id=e5b44df13345468b90906e747c09a977, user.id=6e1b83f0ecf54ae88fc0f287046ea728) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:42.108431 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c911c49-f9d2-4cd8-8fc9-69cecf7bf32c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:42.109919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c911c49-f9d2-4cd8-8fc9-69cecf7bf32c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:42.116098 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 109/224] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:14:41 2022] GET /identity/v3/role_assignments?user.id=6e1b83f0ecf54ae88fc0f287046ea728&scope.project.id=e5b44df13345468b90906e747c09a977&role.id=4912a35790fe4d4297540156083d81d1 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:42.117335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5882a4d-d721-43cf-8c39-5a3cd32e0c30 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:42.117658 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5882a4d-d721-43cf-8c39-5a3cd32e0c30 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:42.117965 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5882a4d-d721-43cf-8c39-5a3cd32e0c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:42.122579 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:42.145815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f5882a4d-d721-43cf-8c39-5a3cd32e0c30 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:42.147275 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f5882a4d-d721-43cf-8c39-5a3cd32e0c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:42.234065 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 110/225] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:42.256867 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5882a4d-d721-43cf-8c39-5a3cd32e0c30 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:42.282005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b2226b-78c2-46cd-9b5b-7670b8f2857b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:42.326947 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f575683-0d9c-424c-813e-f42841dd05e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:42.327974 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f575683-0d9c-424c-813e-f42841dd05e1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:42.327974 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f575683-0d9c-424c-813e-f42841dd05e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:42.328138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f575683-0d9c-424c-813e-f42841dd05e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:42.328904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f575683-0d9c-424c-813e-f42841dd05e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:42.329211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f575683-0d9c-424c-813e-f42841dd05e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:42.331632 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f575683-0d9c-424c-813e-f42841dd05e1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:42.345578 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 116/226] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:42.353628 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f575683-0d9c-424c-813e-f42841dd05e1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:42.466926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f575683-0d9c-424c-813e-f42841dd05e1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:42.469815 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:42.485993 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5882a4d-d721-43cf-8c39-5a3cd32e0c30 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:42.507290 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e810bff-f356-45f0-a683-3dd99f562cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:42.508276 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:42.509258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e810bff-f356-45f0-a683-3dd99f562cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:42.509723 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e810bff-f356-45f0-a683-3dd99f562cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:42.510203 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e810bff-f356-45f0-a683-3dd99f562cd9 None admin] PATH_INFO: `/v3/users/157e0b33fb04490fbea290fbd4683c3f` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:42.512290 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e810bff-f356-45f0-a683-3dd99f562cd9 None admin] RBAC: Authorizing `identity:get_user(user_id=157e0b33fb04490fbea290fbd4683c3f)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:42.514984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e810bff-f356-45f0-a683-3dd99f562cd9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:42.517322 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 117/227] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:42 2022] GET /identity/v3/users/157e0b33fb04490fbea290fbd4683c3f => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:42.523000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e810bff-f356-45f0-a683-3dd99f562cd9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:42.566189 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e72304-66e3-474a-a2be-cf055479df04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:42.566849 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e72304-66e3-474a-a2be-cf055479df04 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:42.567129 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e72304-66e3-474a-a2be-cf055479df04 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:42.567404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e72304-66e3-474a-a2be-cf055479df04 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:42.593442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e72304-66e3-474a-a2be-cf055479df04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:42.598530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-77e72304-66e3-474a-a2be-cf055479df04 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:14:42.598530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:42.598530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:42.598530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:42.598530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:42.598530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:42.598530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:42.598530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:42.598530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:42.598530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:42.598530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:42.598530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:42.598530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:42.598530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:42.598530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:42.598530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:42.599405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:42.599405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:42.599405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:42.599405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:42.599405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:42.599405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:42.599405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:42.599405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:42.599405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:42.599405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:42.599405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:42.599405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:42.599405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:42.599405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:42.599405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:42.599405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:42.600241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:42.600241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:42.600241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:42.600241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:42.600241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:42.600241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:42.600241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:42.600241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:42.600241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:42.600241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:42.600241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:14:42.600241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:42.609558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 111/228] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:14:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:42.610551 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e72304-66e3-474a-a2be-cf055479df04 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:42.715424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e810bff-f356-45f0-a683-3dd99f562cd9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:42.784740 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e72304-66e3-474a-a2be-cf055479df04 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:42.814428 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857526a-35bb-4c57-b865-48a79aa4738e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:42.815985 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857526a-35bb-4c57-b865-48a79aa4738e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:42.816604 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857526a-35bb-4c57-b865-48a79aa4738e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:42.818063 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857526a-35bb-4c57-b865-48a79aa4738e None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:42.819543 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857526a-35bb-4c57-b865-48a79aa4738e None admin] RBAC: Authorizing `identity:get_project(project_id=e5b44df13345468b90906e747c09a977)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:42.821875 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857526a-35bb-4c57-b865-48a79aa4738e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:42.830219 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 118/229] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:42 2022] GET /identity/v3/projects/e5b44df13345468b90906e747c09a977 => generated 302 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:42.833738 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857526a-35bb-4c57-b865-48a79aa4738e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:42.847575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdb336-aadd-4b3a-8c48-0a7d127b4c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:42.848485 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdb336-aadd-4b3a-8c48-0a7d127b4c46 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:42.848938 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdb336-aadd-4b3a-8c48-0a7d127b4c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:42.849378 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdb336-aadd-4b3a-8c48-0a7d127b4c46 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:42.850352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cdb336-aadd-4b3a-8c48-0a7d127b4c46 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:42.850829 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cdb336-aadd-4b3a-8c48-0a7d127b4c46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:42.852414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cdb336-aadd-4b3a-8c48-0a7d127b4c46 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:42.862525 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 112/230] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:14:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:42.868312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdb336-aadd-4b3a-8c48-0a7d127b4c46 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:42.892685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:42.940514 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:43.012246 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857526a-35bb-4c57-b865-48a79aa4738e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:43.047354 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ff22b-cb2f-4044-9397-60e76eab5fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:43.048365 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ff22b-cb2f-4044-9397-60e76eab5fa4 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:43.048850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ff22b-cb2f-4044-9397-60e76eab5fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:43.049321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465ff22b-cb2f-4044-9397-60e76eab5fa4 None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977/users/157e0b33fb04490fbea290fbd4683c3f/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:43.051524 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ff22b-cb2f-4044-9397-60e76eab5fa4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5b44df13345468b90906e747c09a977, user_id=157e0b33fb04490fbea290fbd4683c3f, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:43.053283 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465ff22b-cb2f-4044-9397-60e76eab5fa4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:43.074976 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 119/231] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:14:42 2022] PUT /identity/v3/projects/e5b44df13345468b90906e747c09a977/users/157e0b33fb04490fbea290fbd4683c3f/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:14:43.079076 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdb336-aadd-4b3a-8c48-0a7d127b4c46 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:43.116646 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451b6e3-87fb-4cac-8168-e88e3fd835d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:43.117709 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451b6e3-87fb-4cac-8168-e88e3fd835d9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:43.118219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451b6e3-87fb-4cac-8168-e88e3fd835d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:43.118690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451b6e3-87fb-4cac-8168-e88e3fd835d9 None admin] PATH_INFO: `/v3/users/157e0b33fb04490fbea290fbd4683c3f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:43.120264 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451b6e3-87fb-4cac-8168-e88e3fd835d9 None admin] RBAC: Authorizing `identity:get_user(user_id=157e0b33fb04490fbea290fbd4683c3f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:43.122969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451b6e3-87fb-4cac-8168-e88e3fd835d9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:43.124991 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 113/232] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:42 2022] GET /identity/v3/users/157e0b33fb04490fbea290fbd4683c3f => generated 283 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:43.131165 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ff22b-cb2f-4044-9397-60e76eab5fa4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:43.259900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:43.277192 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465ff22b-cb2f-4044-9397-60e76eab5fa4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:43.315080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe6067-29e8-496d-b4a2-3e02140497ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:43.315656 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe6067-29e8-496d-b4a2-3e02140497ff None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:43.315941 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe6067-29e8-496d-b4a2-3e02140497ff None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:43.316219 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe6067-29e8-496d-b4a2-3e02140497ff None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:43.317446 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe6067-29e8-496d-b4a2-3e02140497ff None admin] RBAC: Authorizing `identity:get_project(project_id=e5b44df13345468b90906e747c09a977)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:43.318945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe6067-29e8-496d-b4a2-3e02140497ff None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:43.320933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 120/233] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:43 2022] GET /identity/v3/projects/e5b44df13345468b90906e747c09a977 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:43.327412 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451b6e3-87fb-4cac-8168-e88e3fd835d9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:43.433783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451b6e3-87fb-4cac-8168-e88e3fd835d9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:43.438020 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:43.472460 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0cba5-fbea-4ee9-b3f8-e795e0d7460c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:43.473140 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0cba5-fbea-4ee9-b3f8-e795e0d7460c None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:43.473422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0cba5-fbea-4ee9-b3f8-e795e0d7460c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:43.473727 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0cba5-fbea-4ee9-b3f8-e795e0d7460c None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977/users/157e0b33fb04490fbea290fbd4683c3f/roles/4146966c23674402aebb9a5b21cf18ba` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:43.475997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a0cba5-fbea-4ee9-b3f8-e795e0d7460c None admin] RBAC: Authorizing `identity:create_grant(project_id=e5b44df13345468b90906e747c09a977, user_id=157e0b33fb04490fbea290fbd4683c3f, role_id=4146966c23674402aebb9a5b21cf18ba)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:43.477463 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a0cba5-fbea-4ee9-b3f8-e795e0d7460c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:43.497206 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 114/234] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:14:43 2022] PUT /identity/v3/projects/e5b44df13345468b90906e747c09a977/users/157e0b33fb04490fbea290fbd4683c3f/roles/4146966c23674402aebb9a5b21cf18ba => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:14:43.630321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488415e1-7113-4f7c-b820-1c9249325dd5 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:43.630716 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488415e1-7113-4f7c-b820-1c9249325dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:43.630944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488415e1-7113-4f7c-b820-1c9249325dd5 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:43.632627 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 121/235] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:43.638375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dc70f-de62-4e50-9bfa-c723040466c8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:43.638913 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dc70f-de62-4e50-9bfa-c723040466c8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:43.638913 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8dc70f-de62-4e50-9bfa-c723040466c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:43.667162 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2a8dc70f-de62-4e50-9bfa-c723040466c8 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:43.669299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2a8dc70f-de62-4e50-9bfa-c723040466c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:43.769959 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 115/236] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:14:43.796055 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488415e1-7113-4f7c-b820-1c9249325dd5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:43.892417 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1742ec-aeab-42e5-adeb-fde2371a9e71 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:43.893108 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1742ec-aeab-42e5-adeb-fde2371a9e71 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:43.893614 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1742ec-aeab-42e5-adeb-fde2371a9e71 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:43.895447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 116/237] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:43.907098 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2a631-4435-4790-a3e6-e8222146060c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:43.907656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2a631-4435-4790-a3e6-e8222146060c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:43.908120 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2a631-4435-4790-a3e6-e8222146060c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:43.929269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488415e1-7113-4f7c-b820-1c9249325dd5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:43.946227 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f3b2a631-4435-4790-a3e6-e8222146060c None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:43.947794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f3b2a631-4435-4790-a3e6-e8222146060c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:44.000987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f1b64-1e08-4823-89b8-a80cf0b588fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:44.002975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f1b64-1e08-4823-89b8-a80cf0b588fc None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:44.004647 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f1b64-1e08-4823-89b8-a80cf0b588fc None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:44.005120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f1b64-1e08-4823-89b8-a80cf0b588fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:44.032295 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f1b64-1e08-4823-89b8-a80cf0b588fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:44.037671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-c81f1b64-1e08-4823-89b8-a80cf0b588fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 22:14:44.037671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:44.037671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:44.037671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:44.037671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:44.037671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:44.037671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:44.037671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:44.037671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:44.037671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:44.037671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:44.037671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:44.037671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:44.037671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:44.037671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:44.037671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:44.038530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:44.038530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:44.038530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:44.038530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:44.038530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:44.038530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:44.038530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:44.038530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:44.038530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:44.038530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:44.038530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:44.038530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:44.038530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:44.038530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:44.038530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:44.038530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:44.039307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:44.039307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:44.039307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:44.039307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:44.039307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:44.039307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:44.039307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:44.039307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:44.039307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:44.039307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:44.039307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 22:14:44.039307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:14:44.046991 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 122/238] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:44.052733 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f1b64-1e08-4823-89b8-a80cf0b588fc None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:44.085359 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:44.102925 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 117/239] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:14:44.170971 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2a631-4435-4790-a3e6-e8222146060c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:44.217310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:44.229513 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f1b64-1e08-4823-89b8-a80cf0b588fc None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:44.290250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216b06c-8004-4f4d-b618-93efd4bbb6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:44.290881 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216b06c-8004-4f4d-b618-93efd4bbb6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:44.291162 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216b06c-8004-4f4d-b618-93efd4bbb6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:44.291437 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c216b06c-8004-4f4d-b618-93efd4bbb6e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:44.293210 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216b06c-8004-4f4d-b618-93efd4bbb6e4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:44.293555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216b06c-8004-4f4d-b618-93efd4bbb6e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:44.295216 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c216b06c-8004-4f4d-b618-93efd4bbb6e4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:44.305597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 123/240] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:44.311064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216b06c-8004-4f4d-b618-93efd4bbb6e4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:44.401316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:44.404767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2a631-4435-4790-a3e6-e8222146060c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:44.432545 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c216b06c-8004-4f4d-b618-93efd4bbb6e4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:44.472102 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a83ba5-1abc-4c31-bea6-1ea462d4a8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:44.473047 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a83ba5-1abc-4c31-bea6-1ea462d4a8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:44.473735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a83ba5-1abc-4c31-bea6-1ea462d4a8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:44.474354 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a83ba5-1abc-4c31-bea6-1ea462d4a8a2 None admin] PATH_INFO: `/v3/users/ccda1ef2f6c745c2b5483d6217503a54` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:44.482347 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ee974-e72d-4778-a6eb-69a62cc6d24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:44.482998 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ee974-e72d-4778-a6eb-69a62cc6d24f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:44.483366 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ee974-e72d-4778-a6eb-69a62cc6d24f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:44.483613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ee974-e72d-4778-a6eb-69a62cc6d24f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:44.485326 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ee974-e72d-4778-a6eb-69a62cc6d24f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:44.487381 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688ee974-e72d-4778-a6eb-69a62cc6d24f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:44.490338 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a83ba5-1abc-4c31-bea6-1ea462d4a8a2 None admin] RBAC: Authorizing `identity:get_user(user_id=ccda1ef2f6c745c2b5483d6217503a54)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:44.491897 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a83ba5-1abc-4c31-bea6-1ea462d4a8a2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:44.493951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 124/242] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:44 2022] GET /identity/v3/users/ccda1ef2f6c745c2b5483d6217503a54 => generated 297 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:44.494092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 118/242] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:14:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:44.495457 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2782e29-8e58-4ea0-8f3b-5cf382f12977 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:44.495836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ee974-e72d-4778-a6eb-69a62cc6d24f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:44.496041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2782e29-8e58-4ea0-8f3b-5cf382f12977 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:44.496663 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2782e29-8e58-4ea0-8f3b-5cf382f12977 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:44.498049 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 125/243] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:44.499338 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2782e29-8e58-4ea0-8f3b-5cf382f12977 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:44.623259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:44.631738 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2782e29-8e58-4ea0-8f3b-5cf382f12977 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:44.637502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688ee974-e72d-4778-a6eb-69a62cc6d24f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:44.678717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a4ae7-f7df-48af-8f8f-ef03407d4e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:44.679511 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a4ae7-f7df-48af-8f8f-ef03407d4e0c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:44.679891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a4ae7-f7df-48af-8f8f-ef03407d4e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:44.680259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a4ae7-f7df-48af-8f8f-ef03407d4e0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:44.682975 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6dda2-e4ba-4fb4-ab75-89b83370a46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:44.684357 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:44.685186 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6dda2-e4ba-4fb4-ab75-89b83370a46e None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:44.685701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6dda2-e4ba-4fb4-ab75-89b83370a46e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:44.686031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6dda2-e4ba-4fb4-ab75-89b83370a46e None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:44.689807 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6dda2-e4ba-4fb4-ab75-89b83370a46e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:44.691276 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e6dda2-e4ba-4fb4-ab75-89b83370a46e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:44.725623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 119/244] 10.222.0.39 () {60 vars in 1132 bytes} [Thu Jul 28 22:14:44 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:14:44.727051 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140324e7-d870-484d-b880-9e7e9eeadd0e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:44.727386 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140324e7-d870-484d-b880-9e7e9eeadd0e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:44.727663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140324e7-d870-484d-b880-9e7e9eeadd0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:44.769140 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-140324e7-d870-484d-b880-9e7e9eeadd0e None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:44.771638 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-140324e7-d870-484d-b880-9e7e9eeadd0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:44.802346 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a4ae7-f7df-48af-8f8f-ef03407d4e0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:44.804182 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a4ae7-f7df-48af-8f8f-ef03407d4e0c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:44.847437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 120/245] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:44.869725 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140324e7-d870-484d-b880-9e7e9eeadd0e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:44.952841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:44.961415 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 126/246] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:14:44.968224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a4ae7-f7df-48af-8f8f-ef03407d4e0c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:44.978618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140324e7-d870-484d-b880-9e7e9eeadd0e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:45.008644 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:45.018360 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38861b-e108-4e8d-9360-1ce748dfc592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:45.019395 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38861b-e108-4e8d-9360-1ce748dfc592 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:45.019866 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38861b-e108-4e8d-9360-1ce748dfc592 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:45.020326 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d38861b-e108-4e8d-9360-1ce748dfc592 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:45.032612 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d38861b-e108-4e8d-9360-1ce748dfc592 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:45.037869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-3d38861b-e108-4e8d-9360-1ce748dfc592 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:45.037869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:45.037869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:45.037869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:45.037869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:45.037869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:45.037869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:45.037869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:45.037869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:45.037869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:45.037869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:45.037869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:45.037869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:45.037869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:45.037869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:45.037869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:45.038830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:45.038830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:45.038830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:45.038830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:45.038830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:45.038830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:45.038830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:45.038830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:45.038830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:45.038830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:45.038830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:45.038830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:45.038830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:45.038830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:45.038830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:45.038830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:45.039633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:45.039633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:45.039633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:45.039633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:45.039633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:45.039633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:45.039633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:45.039633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:45.039633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:45.039633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:45.039633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:45.039633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:45.042287 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 121/247] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:14:45.048776 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38861b-e108-4e8d-9360-1ce748dfc592 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:45.086292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a4ae7-f7df-48af-8f8f-ef03407d4e0c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:45.126468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd79583-ddea-42e4-8dc7-08e76f06d49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:45.127497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd79583-ddea-42e4-8dc7-08e76f06d49c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:45.127997 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd79583-ddea-42e4-8dc7-08e76f06d49c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:45.128476 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd79583-ddea-42e4-8dc7-08e76f06d49c None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:45.130059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd79583-ddea-42e4-8dc7-08e76f06d49c None admin] RBAC: Authorizing `identity:get_project(project_id=e5b44df13345468b90906e747c09a977)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:45.131871 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd79583-ddea-42e4-8dc7-08e76f06d49c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:45.134483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 127/248] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:44 2022] GET /identity/v3/projects/e5b44df13345468b90906e747c09a977 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:45.141042 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd79583-ddea-42e4-8dc7-08e76f06d49c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:45.158489 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d38861b-e108-4e8d-9360-1ce748dfc592 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:45.215669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e11f2d-af1d-4481-88e2-cbdcd43a5dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:45.215870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e11f2d-af1d-4481-88e2-cbdcd43a5dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:45.216222 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e11f2d-af1d-4481-88e2-cbdcd43a5dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:45.216497 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e11f2d-af1d-4481-88e2-cbdcd43a5dd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:45.217217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e11f2d-af1d-4481-88e2-cbdcd43a5dd4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:45.217516 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e11f2d-af1d-4481-88e2-cbdcd43a5dd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:45.218996 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e11f2d-af1d-4481-88e2-cbdcd43a5dd4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:45.221321 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:45.234238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 122/249] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:45.235155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e11f2d-af1d-4481-88e2-cbdcd43a5dd4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:45.288664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd79583-ddea-42e4-8dc7-08e76f06d49c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:45.290711 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:45.330134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ea64d-0e9e-4580-859c-9e98153cd46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:45.331151 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ea64d-0e9e-4580-859c-9e98153cd46f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:45.331687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ea64d-0e9e-4580-859c-9e98153cd46f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:45.332181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ea64d-0e9e-4580-859c-9e98153cd46f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:45.333376 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ea64d-0e9e-4580-859c-9e98153cd46f None admin] RBAC: Adding query filter params (role.id=d2fa7787cf3f461589be9f37afe9de6e, scope.project.id=e5b44df13345468b90906e747c09a977, user.id=ccda1ef2f6c745c2b5483d6217503a54) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:45.333931 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ea64d-0e9e-4580-859c-9e98153cd46f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:45.335675 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ea64d-0e9e-4580-859c-9e98153cd46f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:45.342854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 128/250] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:14:45 2022] GET /identity/v3/role_assignments?user.id=ccda1ef2f6c745c2b5483d6217503a54&scope.project.id=e5b44df13345468b90906e747c09a977&role.id=d2fa7787cf3f461589be9f37afe9de6e => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:45.347743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e11f2d-af1d-4481-88e2-cbdcd43a5dd4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:45.390745 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17646dd-fb2b-47bc-86f2-15f67cfa2fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:45.391791 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17646dd-fb2b-47bc-86f2-15f67cfa2fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:45.392295 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17646dd-fb2b-47bc-86f2-15f67cfa2fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:45.392768 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17646dd-fb2b-47bc-86f2-15f67cfa2fd0 None admin] PATH_INFO: `/v3/users/6e1b83f0ecf54ae88fc0f287046ea728` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:45.394320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17646dd-fb2b-47bc-86f2-15f67cfa2fd0 None admin] RBAC: Authorizing `identity:get_user(user_id=6e1b83f0ecf54ae88fc0f287046ea728)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:45.396042 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17646dd-fb2b-47bc-86f2-15f67cfa2fd0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:45.398181 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 123/251] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:45 2022] GET /identity/v3/users/6e1b83f0ecf54ae88fc0f287046ea728 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:45.405668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ea64d-0e9e-4580-859c-9e98153cd46f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:45.526696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ea64d-0e9e-4580-859c-9e98153cd46f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:45.565177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ebf30-4c5e-4214-8aea-75432c11dc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:45.566273 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ebf30-4c5e-4214-8aea-75432c11dc51 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:45.566801 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ebf30-4c5e-4214-8aea-75432c11dc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:45.567274 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ebf30-4c5e-4214-8aea-75432c11dc51 None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:45.568693 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ebf30-4c5e-4214-8aea-75432c11dc51 None admin] RBAC: Authorizing `identity:get_project(project_id=e5b44df13345468b90906e747c09a977)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:45.570383 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063ebf30-4c5e-4214-8aea-75432c11dc51 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:45.572736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 129/252] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:45 2022] GET /identity/v3/projects/e5b44df13345468b90906e747c09a977 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:45.578165 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17646dd-fb2b-47bc-86f2-15f67cfa2fd0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:45.659284 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:45.699685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e83b8-2619-4710-9740-cd0106690f98 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:45.700422 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e83b8-2619-4710-9740-cd0106690f98 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:45.701363 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79e83b8-2619-4710-9740-cd0106690f98 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:45.704243 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17646dd-fb2b-47bc-86f2-15f67cfa2fd0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:45.704808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 130/253] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:45.712132 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bfdbc-2876-4405-b5b8-9d02dc49c0c0 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:45.713230 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bfdbc-2876-4405-b5b8-9d02dc49c0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:45.713744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967bfdbc-2876-4405-b5b8-9d02dc49c0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:45.739729 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-967bfdbc-2876-4405-b5b8-9d02dc49c0c0 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:45.741281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-967bfdbc-2876-4405-b5b8-9d02dc49c0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:45.748809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7b9e6-3593-4d1d-b7d0-01e683994a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:45.749495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7b9e6-3593-4d1d-b7d0-01e683994a89 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:45.749831 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7b9e6-3593-4d1d-b7d0-01e683994a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:45.750110 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf7b9e6-3593-4d1d-b7d0-01e683994a89 None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977/users/6e1b83f0ecf54ae88fc0f287046ea728/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:45.752133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf7b9e6-3593-4d1d-b7d0-01e683994a89 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5b44df13345468b90906e747c09a977, user_id=6e1b83f0ecf54ae88fc0f287046ea728, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:45.753638 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf7b9e6-3593-4d1d-b7d0-01e683994a89 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:45.770468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 124/254] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:14:45 2022] PUT /identity/v3/projects/e5b44df13345468b90906e747c09a977/users/6e1b83f0ecf54ae88fc0f287046ea728/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:14:45.783003 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:45.848274 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 131/255] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:45.889157 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7b9e6-3593-4d1d-b7d0-01e683994a89 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:45.942405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9359bbd-1578-470e-b3c0-fffff525ceed None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:45.943054 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9359bbd-1578-470e-b3c0-fffff525ceed None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:45.943526 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9359bbd-1578-470e-b3c0-fffff525ceed None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:45.945317 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 132/256] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:45.951890 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfbcb3-271d-4adf-8111-228a1975fb12 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:45.952491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfbcb3-271d-4adf-8111-228a1975fb12 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:45.952963 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfbcb3-271d-4adf-8111-228a1975fb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:45.998642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-4ebfbcb3-271d-4adf-8111-228a1975fb12 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:46.000206 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4ebfbcb3-271d-4adf-8111-228a1975fb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:46.050863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf7b9e6-3593-4d1d-b7d0-01e683994a89 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:46.101890 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:46.115678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb9676-0940-4655-bd8b-aa9fe10079e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:46.118258 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb9676-0940-4655-bd8b-aa9fe10079e5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:46.118730 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb9676-0940-4655-bd8b-aa9fe10079e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:46.119169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb9676-0940-4655-bd8b-aa9fe10079e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:46.140252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb9676-0940-4655-bd8b-aa9fe10079e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:46.143744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 133/257] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:14:46.146307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-4bbb9676-0940-4655-bd8b-aa9fe10079e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:14:46.146307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:46.146307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:46.146307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:46.146307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:46.146307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:46.146307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:46.146307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:46.146307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:46.146307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:46.146307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:46.146307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:46.146307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:46.146307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:46.146307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:46.146307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:46.147405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:46.147405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:46.147405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:46.147405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:46.147405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:46.147405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:46.147405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:46.147405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:46.147405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:46.147405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:46.147405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:46.147405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:46.147405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:46.147405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:46.147405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:46.147405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:46.148385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:46.148385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:46.148385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:46.148385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:46.148385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:46.148385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:46.148385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:46.148385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:46.148385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:46.148385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:46.148385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:14:46.148385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:46.150706 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 125/258] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:14:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:46.157557 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfbcb3-271d-4adf-8111-228a1975fb12 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:46.188428 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb9676-0940-4655-bd8b-aa9fe10079e5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:46.289547 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfbcb3-271d-4adf-8111-228a1975fb12 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:46.293051 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:46.366280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e41c3-d06a-44cf-98e0-39af15f527ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:46.366938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e41c3-d06a-44cf-98e0-39af15f527ae None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:46.367223 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e41c3-d06a-44cf-98e0-39af15f527ae None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:46.367500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e41c3-d06a-44cf-98e0-39af15f527ae None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:46.368240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e41c3-d06a-44cf-98e0-39af15f527ae None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:46.368547 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e41c3-d06a-44cf-98e0-39af15f527ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:46.372352 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e41c3-d06a-44cf-98e0-39af15f527ae None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:46.393486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 134/259] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:46.399450 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e41c3-d06a-44cf-98e0-39af15f527ae None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:46.407795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb9676-0940-4655-bd8b-aa9fe10079e5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:46.481128 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c094578-ff25-41a8-afde-01e7686ac0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:46.481817 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c094578-ff25-41a8-afde-01e7686ac0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:46.482100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c094578-ff25-41a8-afde-01e7686ac0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:46.482375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c094578-ff25-41a8-afde-01e7686ac0a4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:46.505165 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c094578-ff25-41a8-afde-01e7686ac0a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:46.510674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-4c094578-ff25-41a8-afde-01e7686ac0a4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:14:46.510674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:46.510674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:46.510674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:46.510674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:46.510674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:46.510674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:46.510674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:46.510674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:46.510674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:46.510674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:46.510674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:46.510674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:46.510674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:46.510674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:46.510674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:46.512242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:46.512242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:46.512242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:46.512242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:46.512242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:46.512242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:46.512242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:46.512242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:46.512242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:46.512242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:46.512242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:46.512242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:46.512242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:46.512242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:46.512242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:46.512242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:46.513259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:46.513259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:46.513259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:46.513259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:46.513259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:46.513259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:46.513259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:46.513259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:46.513259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:46.513259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:46.513259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:14:46.513259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:46.521912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 126/260] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:14:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:14:46.522811 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c094578-ff25-41a8-afde-01e7686ac0a4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:46.600755 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e41c3-d06a-44cf-98e0-39af15f527ae None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:46.623985 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:46.677092 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62dfd6-656d-4c3b-a718-7fb9f68c739e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:46.677792 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62dfd6-656d-4c3b-a718-7fb9f68c739e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:46.678065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62dfd6-656d-4c3b-a718-7fb9f68c739e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:46.678339 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62dfd6-656d-4c3b-a718-7fb9f68c739e None admin] PATH_INFO: `/v3/users/157e0b33fb04490fbea290fbd4683c3f` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:46.681698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62dfd6-656d-4c3b-a718-7fb9f68c739e None admin] RBAC: Authorizing `identity:get_user(user_id=157e0b33fb04490fbea290fbd4683c3f)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:46.683769 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62dfd6-656d-4c3b-a718-7fb9f68c739e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:46.685656 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 135/261] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:46 2022] GET /identity/v3/users/157e0b33fb04490fbea290fbd4683c3f => generated 283 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:46.691528 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62dfd6-656d-4c3b-a718-7fb9f68c739e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:46.747332 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c094578-ff25-41a8-afde-01e7686ac0a4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:46.758499 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:46.822393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9d363-cf06-4e5f-b141-a1b8fe41fba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:46.823042 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9d363-cf06-4e5f-b141-a1b8fe41fba7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:46.823346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9d363-cf06-4e5f-b141-a1b8fe41fba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:46.824123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e9d363-cf06-4e5f-b141-a1b8fe41fba7 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:46.825249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9d363-cf06-4e5f-b141-a1b8fe41fba7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:46.825665 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9d363-cf06-4e5f-b141-a1b8fe41fba7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:46.827098 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e9d363-cf06-4e5f-b141-a1b8fe41fba7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:46.838592 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 127/262] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:14:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:46.845024 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9d363-cf06-4e5f-b141-a1b8fe41fba7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:46.914543 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62dfd6-656d-4c3b-a718-7fb9f68c739e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:46.979163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eac4f0-dc24-4129-ad99-61849e31bb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:46.980315 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eac4f0-dc24-4129-ad99-61849e31bb23 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:46.980648 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eac4f0-dc24-4129-ad99-61849e31bb23 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:46.980972 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eac4f0-dc24-4129-ad99-61849e31bb23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:47.105332 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eac4f0-dc24-4129-ad99-61849e31bb23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:47.107022 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eac4f0-dc24-4129-ad99-61849e31bb23 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:47.107703 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e9d363-cf06-4e5f-b141-a1b8fe41fba7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:47.127673 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:47.169901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:47.207585 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1101a9-ae37-438f-bb6c-544325f51430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:47.208245 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1101a9-ae37-438f-bb6c-544325f51430 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:47.208527 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1101a9-ae37-438f-bb6c-544325f51430 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:47.208863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1101a9-ae37-438f-bb6c-544325f51430 None admin] PATH_INFO: `/v3/users/157e0b33fb04490fbea290fbd4683c3f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:47.214287 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1101a9-ae37-438f-bb6c-544325f51430 None admin] RBAC: Authorizing `identity:get_user(user_id=157e0b33fb04490fbea290fbd4683c3f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:47.216131 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1101a9-ae37-438f-bb6c-544325f51430 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:47.218121 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 128/263] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:46 2022] GET /identity/v3/users/157e0b33fb04490fbea290fbd4683c3f => generated 283 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:47.223463 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1101a9-ae37-438f-bb6c-544325f51430 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:47.274476 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 136/264] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 22:14:47.280531 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eac4f0-dc24-4129-ad99-61849e31bb23 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:47.452212 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eac4f0-dc24-4129-ad99-61849e31bb23 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:47.500586 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1101a9-ae37-438f-bb6c-544325f51430 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:47.502359 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:47.517132 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357c255-73e6-41ef-95db-00eec10ac861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:47.517852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357c255-73e6-41ef-95db-00eec10ac861 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:47.518141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357c255-73e6-41ef-95db-00eec10ac861 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:47.518871 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357c255-73e6-41ef-95db-00eec10ac861 None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:47.521072 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357c255-73e6-41ef-95db-00eec10ac861 None admin] RBAC: Authorizing `identity:get_project(project_id=e5b44df13345468b90906e747c09a977)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:47.522580 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357c255-73e6-41ef-95db-00eec10ac861 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:47.529719 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 137/265] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:47 2022] GET /identity/v3/projects/e5b44df13345468b90906e747c09a977 => generated 302 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:47.531106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357c255-73e6-41ef-95db-00eec10ac861 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:47.597873 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaeac4d-e982-4868-9b31-e7e1c2b0df53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:47.598510 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaeac4d-e982-4868-9b31-e7e1c2b0df53 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:47.598794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaeac4d-e982-4868-9b31-e7e1c2b0df53 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:47.599074 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaeac4d-e982-4868-9b31-e7e1c2b0df53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:47.710231 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:47.730570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357c255-73e6-41ef-95db-00eec10ac861 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:47.758621 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaeac4d-e982-4868-9b31-e7e1c2b0df53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:47.760200 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaeac4d-e982-4868-9b31-e7e1c2b0df53 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:47.798273 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f5d21-6cf9-42bd-8d0a-62b217524f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:47.798907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f5d21-6cf9-42bd-8d0a-62b217524f1d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:47.799191 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f5d21-6cf9-42bd-8d0a-62b217524f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:47.799465 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624f5d21-6cf9-42bd-8d0a-62b217524f1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:47.800268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f5d21-6cf9-42bd-8d0a-62b217524f1d None admin] RBAC: Adding query filter params (role.id=aded7bd43e974686b3c23bd9d4255b24, scope.project.id=e5b44df13345468b90906e747c09a977, user.id=157e0b33fb04490fbea290fbd4683c3f) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:47.800579 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f5d21-6cf9-42bd-8d0a-62b217524f1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:47.802080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624f5d21-6cf9-42bd-8d0a-62b217524f1d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:47.812604 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 138/266] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:14:47 2022] GET /identity/v3/role_assignments?user.id=157e0b33fb04490fbea290fbd4683c3f&scope.project.id=e5b44df13345468b90906e747c09a977&role.id=aded7bd43e974686b3c23bd9d4255b24 => generated 608 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:47.962542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 129/267] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 739 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 22:14:47.969388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f5d21-6cf9-42bd-8d0a-62b217524f1d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:47.979131 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867434a-7b51-4d08-aec6-31c6ff5dcc1b None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:47.979421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867434a-7b51-4d08-aec6-31c6ff5dcc1b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:47.979698 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867434a-7b51-4d08-aec6-31c6ff5dcc1b None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:47.982493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 130/268] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:14:47.987536 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d380fb4-618b-480b-81a9-893fd5eb905b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:47.987819 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d380fb4-618b-480b-81a9-893fd5eb905b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:47.988093 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d380fb4-618b-480b-81a9-893fd5eb905b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:48.006505 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1d380fb4-618b-480b-81a9-893fd5eb905b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:48.007711 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1d380fb4-618b-480b-81a9-893fd5eb905b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:48.112301 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:48.152009 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 131/269] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:14:48.181359 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624f5d21-6cf9-42bd-8d0a-62b217524f1d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:48.199589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d380fb4-618b-480b-81a9-893fd5eb905b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:48.211644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:48.224333 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bb572-02e8-4854-a1b3-61fcc182bd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:48.225450 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bb572-02e8-4854-a1b3-61fcc182bd6a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:48.225906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bb572-02e8-4854-a1b3-61fcc182bd6a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:48.226388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bb572-02e8-4854-a1b3-61fcc182bd6a None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:48.227757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381bb572-02e8-4854-a1b3-61fcc182bd6a None admin] RBAC: Authorizing `identity:get_project(project_id=e5b44df13345468b90906e747c09a977)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:48.229421 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381bb572-02e8-4854-a1b3-61fcc182bd6a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:48.232488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 139/270] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:47 2022] GET /identity/v3/projects/e5b44df13345468b90906e747c09a977 => generated 302 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:48.239645 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bb572-02e8-4854-a1b3-61fcc182bd6a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:48.362320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bb572-02e8-4854-a1b3-61fcc182bd6a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:48.409886 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:48.430187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d380fb4-618b-480b-81a9-893fd5eb905b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:48.435181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebbd00-1371-4350-9410-e60e759f18c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:48.435955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebbd00-1371-4350-9410-e60e759f18c3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:48.436340 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebbd00-1371-4350-9410-e60e759f18c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:48.436713 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebbd00-1371-4350-9410-e60e759f18c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:48.437650 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebbd00-1371-4350-9410-e60e759f18c3 None admin] RBAC: Adding query filter params (role.id=4146966c23674402aebb9a5b21cf18ba, scope.project.id=e5b44df13345468b90906e747c09a977, user.id=157e0b33fb04490fbea290fbd4683c3f) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:48.438064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebbd00-1371-4350-9410-e60e759f18c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:48.439611 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caebbd00-1371-4350-9410-e60e759f18c3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:48.450436 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 140/271] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:14:48 2022] GET /identity/v3/role_assignments?user.id=157e0b33fb04490fbea290fbd4683c3f&scope.project.id=e5b44df13345468b90906e747c09a977&role.id=4146966c23674402aebb9a5b21cf18ba => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:48.469168 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29774d41-e364-459a-a0a2-2b823b8f20b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:48.469868 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29774d41-e364-459a-a0a2-2b823b8f20b5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:48.470192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29774d41-e364-459a-a0a2-2b823b8f20b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:48.470541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29774d41-e364-459a-a0a2-2b823b8f20b5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:48.487358 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29774d41-e364-459a-a0a2-2b823b8f20b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:48.492513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-29774d41-e364-459a-a0a2-2b823b8f20b5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 22:14:48.492513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:48.492513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:48.492513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:48.492513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:48.492513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:48.492513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:48.492513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:48.492513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:48.492513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:48.492513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:48.492513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:48.492513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:48.492513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:48.492513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:48.492513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:48.493406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:48.493406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:48.493406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:48.493406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:48.493406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:48.493406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:48.493406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:48.493406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:48.493406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:48.493406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:48.493406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:48.493406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:48.493406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:48.493406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:48.493406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:48.493406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:48.494509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:48.494509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:48.494509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:48.494509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:48.494509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:48.494509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:48.494509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:48.494509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:48.494509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:48.494509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:48.494509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 22:14:48.494509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:48.494509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 132/272] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:48.499975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebbd00-1371-4350-9410-e60e759f18c3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:48.632608 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebbd00-1371-4350-9410-e60e759f18c3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:48.645737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b03a8-436e-445a-b4b7-4fcc94b4347b None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:48.646301 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b03a8-436e-445a-b4b7-4fcc94b4347b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:48.646801 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79b03a8-436e-445a-b4b7-4fcc94b4347b None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:48.648343 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 133/273] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:14:48.659463 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0372e3-5b2a-43aa-b3e3-99ef1e2bfb6b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:48.660060 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0372e3-5b2a-43aa-b3e3-99ef1e2bfb6b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:48.660509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0372e3-5b2a-43aa-b3e3-99ef1e2bfb6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:48.684807 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7c0372e3-5b2a-43aa-b3e3-99ef1e2bfb6b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:48.686062 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7c0372e3-5b2a-43aa-b3e3-99ef1e2bfb6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:48.702985 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a6101f-a2d2-43c5-8a55-2d57596ca27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:48.704391 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6101f-a2d2-43c5-8a55-2d57596ca27b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:48.705126 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6101f-a2d2-43c5-8a55-2d57596ca27b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:48.705640 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a6101f-a2d2-43c5-8a55-2d57596ca27b None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:48.706742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a6101f-a2d2-43c5-8a55-2d57596ca27b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:48.707601 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a6101f-a2d2-43c5-8a55-2d57596ca27b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:48.709229 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a6101f-a2d2-43c5-8a55-2d57596ca27b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:48.723066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 141/274] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:48.724759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0a64c-fb01-4f2d-9932-9ece6e45fb96 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:48.725225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0a64c-fb01-4f2d-9932-9ece6e45fb96 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:48.725690 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0a64c-fb01-4f2d-9932-9ece6e45fb96 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:48.728379 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 142/275] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:48.729105 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0a64c-fb01-4f2d-9932-9ece6e45fb96 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:48.758736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:48.769433 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 134/276] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:48.770743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c414aa7-4918-4601-ac83-0e12da4e4d99 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:48.771024 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c414aa7-4918-4601-ac83-0e12da4e4d99 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:48.771291 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c414aa7-4918-4601-ac83-0e12da4e4d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:48.790843 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2c414aa7-4918-4601-ac83-0e12da4e4d99 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:48.792106 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2c414aa7-4918-4601-ac83-0e12da4e4d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:48.797665 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:48.853585 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0a64c-fb01-4f2d-9932-9ece6e45fb96 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:48.862948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 135/277] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:48.864195 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa57462-6f74-498e-ada1-ae68302a2a86 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:48.864481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa57462-6f74-498e-ada1-ae68302a2a86 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:48.864756 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa57462-6f74-498e-ada1-ae68302a2a86 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:48.866030 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 136/278] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:48.869867 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa57462-6f74-498e-ada1-ae68302a2a86 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:48.892483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebfd64-f7dd-4669-ac89-55bbd3dbb04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:48.893125 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebfd64-f7dd-4669-ac89-55bbd3dbb04a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:48.893422 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebfd64-f7dd-4669-ac89-55bbd3dbb04a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:48.893722 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebfd64-f7dd-4669-ac89-55bbd3dbb04a None admin] PATH_INFO: `/v3/users/ccda1ef2f6c745c2b5483d6217503a54` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:48.894887 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ebfd64-f7dd-4669-ac89-55bbd3dbb04a None admin] RBAC: Authorizing `identity:get_user(user_id=ccda1ef2f6c745c2b5483d6217503a54)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:48.896322 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ebfd64-f7dd-4669-ac89-55bbd3dbb04a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:48.897954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 143/279] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:48 2022] GET /identity/v3/users/ccda1ef2f6c745c2b5483d6217503a54 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:48.899061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a5f72-eef2-40c0-b21d-2feef1c8f925 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:48.899345 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a5f72-eef2-40c0-b21d-2feef1c8f925 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:48.899623 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a5f72-eef2-40c0-b21d-2feef1c8f925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:48.917039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-7b1a5f72-eef2-40c0-b21d-2feef1c8f925 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:48.918812 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7b1a5f72-eef2-40c0-b21d-2feef1c8f925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:48.981585 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa57462-6f74-498e-ada1-ae68302a2a86 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:48.985788 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 144/280] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:48.986653 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a5f72-eef2-40c0-b21d-2feef1c8f925 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:49.007893 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:49.017874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17cfbb-86a6-4f4e-9bcf-5f5d81454a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:49.018457 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17cfbb-86a6-4f4e-9bcf-5f5d81454a9e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:49.018739 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17cfbb-86a6-4f4e-9bcf-5f5d81454a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:49.019010 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef17cfbb-86a6-4f4e-9bcf-5f5d81454a9e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:49.029556 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef17cfbb-86a6-4f4e-9bcf-5f5d81454a9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:49.034488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ef17cfbb-86a6-4f4e-9bcf-5f5d81454a9e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:49.034488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:49.034488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:49.034488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:49.034488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:49.034488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:49.034488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:49.034488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:49.034488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:49.034488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:49.034488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:49.034488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:49.034488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:49.034488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:49.034488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:49.034488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:49.035359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:49.035359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:49.035359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:49.035359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:49.035359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:49.035359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:49.035359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:49.035359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:49.035359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:49.035359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:49.035359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:49.035359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:49.035359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:49.035359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:49.035359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:49.035359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:49.036130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:49.036130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:49.036130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:49.036130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:49.036130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:49.036130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:49.036130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:49.036130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:49.036130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:49.036130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:49.036130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:49.036130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:49.036130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 137/281] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:49.036769 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17cfbb-86a6-4f4e-9bcf-5f5d81454a9e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:49.039930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:49.092218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a5f72-eef2-40c0-b21d-2feef1c8f925 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:49.126953 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7dbe7b-3fcb-4627-aa9f-d97f370cb711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:49.127595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7dbe7b-3fcb-4627-aa9f-d97f370cb711 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:49.127879 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7dbe7b-3fcb-4627-aa9f-d97f370cb711 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:49.128158 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7dbe7b-3fcb-4627-aa9f-d97f370cb711 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:49.135821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef17cfbb-86a6-4f4e-9bcf-5f5d81454a9e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:49.138245 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7dbe7b-3fcb-4627-aa9f-d97f370cb711 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:49.143631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-1e7dbe7b-3fcb-4627-aa9f-d97f370cb711 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:49.143631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:49.143631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:49.143631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:49.143631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:49.143631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:49.143631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:49.143631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:49.143631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:49.143631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:49.143631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:49.143631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:49.143631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:49.143631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:49.143631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:49.143631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:49.144537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:49.144537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:49.144537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:49.144537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:49.144537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:49.144537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:49.144537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:49.144537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:49.144537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:49.144537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:49.144537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:49.144537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:49.144537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:49.144537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:49.144537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:49.144537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:49.145347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:49.145347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:49.145347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:49.145347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:49.145347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:49.145347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:49.145347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:49.145347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:49.145347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:49.145347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:49.145347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:49.145347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:14:49.145347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 145/282] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:49.146157 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7dbe7b-3fcb-4627-aa9f-d97f370cb711 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:49.174306 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30553e7-3fdb-4a6c-8164-03bb83c215ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:49.174942 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30553e7-3fdb-4a6c-8164-03bb83c215ab None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:49.175221 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30553e7-3fdb-4a6c-8164-03bb83c215ab None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:49.175487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30553e7-3fdb-4a6c-8164-03bb83c215ab None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:49.176632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30553e7-3fdb-4a6c-8164-03bb83c215ab None admin] RBAC: Authorizing `identity:get_project(project_id=e5b44df13345468b90906e747c09a977)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:49.178241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30553e7-3fdb-4a6c-8164-03bb83c215ab None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:49.179902 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 138/283] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:49 2022] GET /identity/v3/projects/e5b44df13345468b90906e747c09a977 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:49.180656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30553e7-3fdb-4a6c-8164-03bb83c215ab None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:49.239520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:49.249602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7dbe7b-3fcb-4627-aa9f-d97f370cb711 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:49.259040 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:49.284477 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30553e7-3fdb-4a6c-8164-03bb83c215ab None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:49.285029 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4744aa2-fad2-45e5-8d64-4a9a91588f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:49.285715 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4744aa2-fad2-45e5-8d64-4a9a91588f18 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:49.286002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4744aa2-fad2-45e5-8d64-4a9a91588f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:49.286279 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4744aa2-fad2-45e5-8d64-4a9a91588f18 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:49.287465 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4744aa2-fad2-45e5-8d64-4a9a91588f18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:49.288920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4744aa2-fad2-45e5-8d64-4a9a91588f18 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:49.290555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 146/284] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:14:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:49.291320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4744aa2-fad2-45e5-8d64-4a9a91588f18 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:49.320718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce602ed-345c-4420-a6fa-484fa9d026cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:49.321489 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce602ed-345c-4420-a6fa-484fa9d026cb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:49.321801 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce602ed-345c-4420-a6fa-484fa9d026cb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:49.322075 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce602ed-345c-4420-a6fa-484fa9d026cb None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:49.322791 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce602ed-345c-4420-a6fa-484fa9d026cb None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:49.323101 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce602ed-345c-4420-a6fa-484fa9d026cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:49.324524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce602ed-345c-4420-a6fa-484fa9d026cb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:49.332519 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 139/285] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:49.333121 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce602ed-345c-4420-a6fa-484fa9d026cb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:49.395916 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4744aa2-fad2-45e5-8d64-4a9a91588f18 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:49.429245 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2117b8-7bc7-4e8e-9485-077346433166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:49.429891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2117b8-7bc7-4e8e-9485-077346433166 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:49.430181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2117b8-7bc7-4e8e-9485-077346433166 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:49.430458 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2117b8-7bc7-4e8e-9485-077346433166 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:49.431178 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2117b8-7bc7-4e8e-9485-077346433166 None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:49.431488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2117b8-7bc7-4e8e-9485-077346433166 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:49.432937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2117b8-7bc7-4e8e-9485-077346433166 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:49.433473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce602ed-345c-4420-a6fa-484fa9d026cb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:49.441090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 147/286] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:49.441894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2117b8-7bc7-4e8e-9485-077346433166 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:49.453104 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:49.471656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbde9e4-9d8d-4b6c-b837-3632a81e3929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:49.472326 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbde9e4-9d8d-4b6c-b837-3632a81e3929 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:49.472608 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbde9e4-9d8d-4b6c-b837-3632a81e3929 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:49.472884 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbde9e4-9d8d-4b6c-b837-3632a81e3929 None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977/users/ccda1ef2f6c745c2b5483d6217503a54/roles/d2fa7787cf3f461589be9f37afe9de6e` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:49.474868 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbde9e4-9d8d-4b6c-b837-3632a81e3929 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5b44df13345468b90906e747c09a977, user_id=ccda1ef2f6c745c2b5483d6217503a54, role_id=d2fa7787cf3f461589be9f37afe9de6e)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:49.476323 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbde9e4-9d8d-4b6c-b837-3632a81e3929 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:49.481229 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:49.489223 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 140/287] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:14:49 2022] PUT /identity/v3/projects/e5b44df13345468b90906e747c09a977/users/ccda1ef2f6c745c2b5483d6217503a54/roles/d2fa7787cf3f461589be9f37afe9de6e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:14:49.490150 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbde9e4-9d8d-4b6c-b837-3632a81e3929 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:49.554313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2117b8-7bc7-4e8e-9485-077346433166 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:49.592678 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59df34-62bb-4c24-a54a-db7991522893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:49.593315 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59df34-62bb-4c24-a54a-db7991522893 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:49.593625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59df34-62bb-4c24-a54a-db7991522893 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:49.593905 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd59df34-62bb-4c24-a54a-db7991522893 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:49.594605 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59df34-62bb-4c24-a54a-db7991522893 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:49.596203 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd59df34-62bb-4c24-a54a-db7991522893 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:49.619803 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbde9e4-9d8d-4b6c-b837-3632a81e3929 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:49.629962 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 148/288] 10.222.0.39 () {60 vars in 1132 bytes} [Thu Jul 28 22:14:49 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:14:49.629962 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59df34-62bb-4c24-a54a-db7991522893 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:49.683841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e569b-7c37-48cf-be00-653a502cd10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:49.684512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e569b-7c37-48cf-be00-653a502cd10c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:49.684794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e569b-7c37-48cf-be00-653a502cd10c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:49.685089 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e569b-7c37-48cf-be00-653a502cd10c None admin] PATH_INFO: `/v3/users/6e1b83f0ecf54ae88fc0f287046ea728` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:49.693147 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e569b-7c37-48cf-be00-653a502cd10c None admin] RBAC: Authorizing `identity:get_user(user_id=6e1b83f0ecf54ae88fc0f287046ea728)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:49.693337 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e569b-7c37-48cf-be00-653a502cd10c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:49.697634 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 141/289] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:49 2022] GET /identity/v3/users/6e1b83f0ecf54ae88fc0f287046ea728 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:49.704771 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e569b-7c37-48cf-be00-653a502cd10c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:49.771199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:49.776106 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:49.786536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd59df34-62bb-4c24-a54a-db7991522893 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:49.836964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cc8a3-5c71-4f22-8e00-f50fa2ca7a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:49.837727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cc8a3-5c71-4f22-8e00-f50fa2ca7a36 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:49.838072 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cc8a3-5c71-4f22-8e00-f50fa2ca7a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:49.838353 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22cc8a3-5c71-4f22-8e00-f50fa2ca7a36 None admin] PATH_INFO: `/v3/users/2e920774520346ef96f19617348dcc3b` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:49.856849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22cc8a3-5c71-4f22-8e00-f50fa2ca7a36 None admin] RBAC: Authorizing `identity:get_user(user_id=2e920774520346ef96f19617348dcc3b)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:49.858532 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22cc8a3-5c71-4f22-8e00-f50fa2ca7a36 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:49.860844 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 149/290] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:49 2022] GET /identity/v3/users/2e920774520346ef96f19617348dcc3b => generated 293 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:49.862628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e569b-7c37-48cf-be00-653a502cd10c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:49.866245 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cc8a3-5c71-4f22-8e00-f50fa2ca7a36 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:49.907168 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447db097-639a-45f8-9001-0afecafb9907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:49.907797 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447db097-639a-45f8-9001-0afecafb9907 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:49.908427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447db097-639a-45f8-9001-0afecafb9907 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:49.909162 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447db097-639a-45f8-9001-0afecafb9907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:50.005277 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447db097-639a-45f8-9001-0afecafb9907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:50.007252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447db097-639a-45f8-9001-0afecafb9907 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:50.007645 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22cc8a3-5c71-4f22-8e00-f50fa2ca7a36 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:50.056736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9aed2-032c-437c-9c9d-3dee7de77260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:50.057640 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9aed2-032c-437c-9c9d-3dee7de77260 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:50.059623 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9aed2-032c-437c-9c9d-3dee7de77260 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:50.059901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9aed2-032c-437c-9c9d-3dee7de77260 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:50.060718 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9aed2-032c-437c-9c9d-3dee7de77260 None admin] RBAC: Adding query filter params (role.id=4912a35790fe4d4297540156083d81d1, user.id=2e920774520346ef96f19617348dcc3b, scope.system=all) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:50.061029 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9aed2-032c-437c-9c9d-3dee7de77260 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:50.062575 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9aed2-032c-437c-9c9d-3dee7de77260 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:50.070461 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:50.070605 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 150/291] 10.222.0.39 () {56 vars in 1308 bytes} [Thu Jul 28 22:14:49 2022] GET /identity/v3/role_assignments?user.id=2e920774520346ef96f19617348dcc3b&scope.system=all&role.id=4912a35790fe4d4297540156083d81d1 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:50.142545 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 142/292] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:14:50.148242 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9aed2-032c-437c-9c9d-3dee7de77260 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:50.193556 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:50.291285 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9aed2-032c-437c-9c9d-3dee7de77260 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:50.327924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39773b47-64c4-43c9-860e-8570f06ede8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:50.328858 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39773b47-64c4-43c9-860e-8570f06ede8a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:50.329311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39773b47-64c4-43c9-860e-8570f06ede8a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:50.329810 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39773b47-64c4-43c9-860e-8570f06ede8a None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:50.331148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39773b47-64c4-43c9-860e-8570f06ede8a None admin] RBAC: Authorizing `identity:get_project(project_id=e5b44df13345468b90906e747c09a977)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:50.332809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39773b47-64c4-43c9-860e-8570f06ede8a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:50.339925 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 151/293] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:50 2022] GET /identity/v3/projects/e5b44df13345468b90906e747c09a977 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:50.341398 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447db097-639a-45f8-9001-0afecafb9907 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:50.465519 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447db097-639a-45f8-9001-0afecafb9907 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:50.515488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2bed8-b318-4ff3-b71c-0787bf142fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:50.516117 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2bed8-b318-4ff3-b71c-0787bf142fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:50.516395 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2bed8-b318-4ff3-b71c-0787bf142fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:50.516661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2bed8-b318-4ff3-b71c-0787bf142fc2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:50.517451 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2bed8-b318-4ff3-b71c-0787bf142fc2 None admin] RBAC: Adding query filter params (role.id=4912a35790fe4d4297540156083d81d1, scope.project.id=e5b44df13345468b90906e747c09a977, user.id=6e1b83f0ecf54ae88fc0f287046ea728) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:50.517780 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2bed8-b318-4ff3-b71c-0787bf142fc2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:50.519221 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2bed8-b318-4ff3-b71c-0787bf142fc2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:50.529316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 143/294] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:14:50 2022] GET /identity/v3/role_assignments?user.id=6e1b83f0ecf54ae88fc0f287046ea728&scope.project.id=e5b44df13345468b90906e747c09a977&role.id=4912a35790fe4d4297540156083d81d1 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:51.642394 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d0c717-c388-4265-8697-a1aec660c4dc None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:51.642906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d0c717-c388-4265-8697-a1aec660c4dc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:51.642981 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d0c717-c388-4265-8697-a1aec660c4dc None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:51.644576 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 152/295] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:51.649640 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27ed58-47f0-431f-82e0-45def038a50e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:51.650022 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27ed58-47f0-431f-82e0-45def038a50e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:51.650203 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27ed58-47f0-431f-82e0-45def038a50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:51.668561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ae27ed58-47f0-431f-82e0-45def038a50e None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:51.669813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ae27ed58-47f0-431f-82e0-45def038a50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:51.671892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:51.740614 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 144/296] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:51.763607 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d0c717-c388-4265-8697-a1aec660c4dc None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:51.855802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:51.874127 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d0c717-c388-4265-8697-a1aec660c4dc None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:51.913775 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227078d-64e3-42b7-8736-dd4478249655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:51.914341 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227078d-64e3-42b7-8736-dd4478249655 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:51.914618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227078d-64e3-42b7-8736-dd4478249655 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:51.914894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227078d-64e3-42b7-8736-dd4478249655 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:51.925816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227078d-64e3-42b7-8736-dd4478249655 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:51.930839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-2227078d-64e3-42b7-8736-dd4478249655 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 22:14:51.930839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:51.930839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:51.930839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:51.930839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:51.930839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:51.930839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:51.930839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:51.930839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:51.930839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:51.930839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:51.930839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:51.930839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:51.930839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:51.930839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:51.930839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:51.931727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:51.931727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:51.931727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:51.931727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:51.931727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:51.931727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:51.931727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:51.931727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:51.931727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:51.931727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:51.931727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:51.931727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:51.931727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:51.931727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:51.931727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:51.931727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:51.932510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:51.932510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:51.932510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:51.932510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:51.932510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:51.932510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:51.932510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:51.932510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:51.932510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:51.932510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:51.932510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 22:14:51.932510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:14:51.932510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 153/297] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:51.938573 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27ed58-47f0-431f-82e0-45def038a50e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:51.949411 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585ea3d-c7e9-4912-b9be-84650b2edb90 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:51.950067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585ea3d-c7e9-4912-b9be-84650b2edb90 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:51.950592 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f585ea3d-c7e9-4912-b9be-84650b2edb90 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:51.952536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 154/298] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:14:51.960290 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37088955-01ca-4f73-a880-c277f432cce2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:51.960835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37088955-01ca-4f73-a880-c277f432cce2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:51.961307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37088955-01ca-4f73-a880-c277f432cce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:51.981078 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-37088955-01ca-4f73-a880-c277f432cce2 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:51.982033 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-37088955-01ca-4f73-a880-c277f432cce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:52.064063 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 155/299] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:14:52.069678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27ed58-47f0-431f-82e0-45def038a50e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:52.088119 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37088955-01ca-4f73-a880-c277f432cce2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:52.103809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757537b2-01bf-4cdf-a2d5-788d8858031c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:52.104439 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757537b2-01bf-4cdf-a2d5-788d8858031c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:52.105476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757537b2-01bf-4cdf-a2d5-788d8858031c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:52.105801 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757537b2-01bf-4cdf-a2d5-788d8858031c None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:52.106527 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757537b2-01bf-4cdf-a2d5-788d8858031c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:52.106839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757537b2-01bf-4cdf-a2d5-788d8858031c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:52.109038 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757537b2-01bf-4cdf-a2d5-788d8858031c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:52.116082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:52.122004 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 145/300] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:52.127462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757537b2-01bf-4cdf-a2d5-788d8858031c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:52.179559 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:52.207526 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37088955-01ca-4f73-a880-c277f432cce2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:52.237193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757537b2-01bf-4cdf-a2d5-788d8858031c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:52.252372 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6e301-77de-4f40-bcfb-398ce19f8914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:52.253248 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6e301-77de-4f40-bcfb-398ce19f8914 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:52.253366 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6e301-77de-4f40-bcfb-398ce19f8914 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:52.253576 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6e301-77de-4f40-bcfb-398ce19f8914 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:52.264402 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6e301-77de-4f40-bcfb-398ce19f8914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:52.270211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-b9f6e301-77de-4f40-bcfb-398ce19f8914 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 22:14:52.270211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:52.270211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:52.270211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:52.270211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:52.270211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:52.270211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:52.270211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:52.270211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:52.270211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:52.270211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:52.270211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:52.270211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:52.270211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:52.270211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:52.270211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:52.271077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:52.271077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:52.271077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:52.271077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:52.271077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:52.271077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:52.271077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:52.271077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:52.271077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:52.271077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:52.271077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:52.271077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:52.271077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:52.271077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:52.271077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:52.271077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:52.276555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:52.276555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:52.276555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:52.276555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:52.276555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:52.276555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:52.276555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:52.276555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:52.276555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:52.276555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:52.276555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 22:14:52.276555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:14:52.276555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 156/301] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:14:52.276555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b552d-d1ec-4cc0-a3e7-f035b2d5d54d None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:52.277269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b552d-d1ec-4cc0-a3e7-f035b2d5d54d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:52.277269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b552d-d1ec-4cc0-a3e7-f035b2d5d54d None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:52.277269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 157/302] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:52.278957 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b552d-d1ec-4cc0-a3e7-f035b2d5d54d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:52.281184 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18e2c3-3795-4841-bcc0-114a0d7b509f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:52.281871 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18e2c3-3795-4841-bcc0-114a0d7b509f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:52.282164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18e2c3-3795-4841-bcc0-114a0d7b509f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:52.282605 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18e2c3-3795-4841-bcc0-114a0d7b509f None admin] PATH_INFO: `/v3/users/ccda1ef2f6c745c2b5483d6217503a54` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:52.284647 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18e2c3-3795-4841-bcc0-114a0d7b509f None admin] RBAC: Authorizing `identity:get_user(user_id=ccda1ef2f6c745c2b5483d6217503a54)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:52.286127 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18e2c3-3795-4841-bcc0-114a0d7b509f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:52.287878 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 146/303] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:52 2022] GET /identity/v3/users/ccda1ef2f6c745c2b5483d6217503a54 => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:52.290738 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eab04a-ccbd-4ae9-91f6-0c574d76f3db None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:52.291024 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eab04a-ccbd-4ae9-91f6-0c574d76f3db None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:52.291299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eab04a-ccbd-4ae9-91f6-0c574d76f3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:52.312177 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-79eab04a-ccbd-4ae9-91f6-0c574d76f3db None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:52.313426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-79eab04a-ccbd-4ae9-91f6-0c574d76f3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:52.382409 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:52.389233 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 147/304] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:52.398647 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eab04a-ccbd-4ae9-91f6-0c574d76f3db None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:52.400341 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5b552d-d1ec-4cc0-a3e7-f035b2d5d54d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:52.441519 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124926e-03e7-4a4a-8d02-c3f9600a86e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:52.442092 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124926e-03e7-4a4a-8d02-c3f9600a86e3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:52.442376 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124926e-03e7-4a4a-8d02-c3f9600a86e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:52.442652 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9124926e-03e7-4a4a-8d02-c3f9600a86e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:52.443369 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9124926e-03e7-4a4a-8d02-c3f9600a86e3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:52.443674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9124926e-03e7-4a4a-8d02-c3f9600a86e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:52.445103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9124926e-03e7-4a4a-8d02-c3f9600a86e3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:52.446574 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:52.458322 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 158/305] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:52.458322 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124926e-03e7-4a4a-8d02-c3f9600a86e3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:52.515700 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eab04a-ccbd-4ae9-91f6-0c574d76f3db None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:52.552864 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ad1ed5-b519-4ccb-9e2d-8e92f6daf442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:52.553488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad1ed5-b519-4ccb-9e2d-8e92f6daf442 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:52.553831 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad1ed5-b519-4ccb-9e2d-8e92f6daf442 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:52.554105 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ad1ed5-b519-4ccb-9e2d-8e92f6daf442 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:52.561411 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9124926e-03e7-4a4a-8d02-c3f9600a86e3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:52.593852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cafeb-9372-4462-ba76-0202146ad473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:52.594482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cafeb-9372-4462-ba76-0202146ad473 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:52.594760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cafeb-9372-4462-ba76-0202146ad473 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:52.595029 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cafeb-9372-4462-ba76-0202146ad473 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:52.596163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919cafeb-9372-4462-ba76-0202146ad473 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:52.597624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919cafeb-9372-4462-ba76-0202146ad473 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:52.599206 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 159/306] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:14:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:52.599954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cafeb-9372-4462-ba76-0202146ad473 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:52.623387 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:52.626755 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad1ed5-b519-4ccb-9e2d-8e92f6daf442 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:52.628636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ad1ed5-b519-4ccb-9e2d-8e92f6daf442 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:52.666753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:52.705108 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cafeb-9372-4462-ba76-0202146ad473 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:52.727871 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 148/307] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:14:52.729132 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df945c34-0172-4318-bc18-2872c881a88b None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:52.730011 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df945c34-0172-4318-bc18-2872c881a88b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:52.730418 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df945c34-0172-4318-bc18-2872c881a88b None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:52.733562 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 149/308] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:52.733562 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df945c34-0172-4318-bc18-2872c881a88b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:52.739164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68404e-fb5e-40de-93e4-4067a3651476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:52.739800 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68404e-fb5e-40de-93e4-4067a3651476 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:52.740081 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68404e-fb5e-40de-93e4-4067a3651476 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:52.740352 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68404e-fb5e-40de-93e4-4067a3651476 None admin] PATH_INFO: `/v3/users/8e51e4ebc9d2480b94753e3b45464112` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:52.753491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68404e-fb5e-40de-93e4-4067a3651476 None admin] RBAC: Authorizing `identity:get_user(user_id=8e51e4ebc9d2480b94753e3b45464112)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:52.754965 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68404e-fb5e-40de-93e4-4067a3651476 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:52.756727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 160/309] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:52 2022] GET /identity/v3/users/8e51e4ebc9d2480b94753e3b45464112 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:52.757330 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68404e-fb5e-40de-93e4-4067a3651476 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:52.830140 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:52.835488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df945c34-0172-4318-bc18-2872c881a88b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:52.857236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68404e-fb5e-40de-93e4-4067a3651476 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:52.869900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef58079-dfd5-4dd2-8b0b-b0dfea87ba50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:52.870566 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef58079-dfd5-4dd2-8b0b-b0dfea87ba50 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:52.870851 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef58079-dfd5-4dd2-8b0b-b0dfea87ba50 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:52.871117 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef58079-dfd5-4dd2-8b0b-b0dfea87ba50 None admin] PATH_INFO: `/v3/groups` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:52.871771 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef58079-dfd5-4dd2-8b0b-b0dfea87ba50 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:52.873200 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef58079-dfd5-4dd2-8b0b-b0dfea87ba50 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:52.879187 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:52.889218 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 150/310] 10.222.0.39 () {60 vars in 1134 bytes} [Thu Jul 28 22:14:52 2022] POST /identity/v3/groups => generated 229 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:14:52.890814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b56547-8eb6-410e-a13e-af7e4c8d3257 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:52.891172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b56547-8eb6-410e-a13e-af7e4c8d3257 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:52.891519 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b56547-8eb6-410e-a13e-af7e4c8d3257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:52.893792 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da5539-68cc-49db-adc8-5c84a41f088e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:52.894633 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da5539-68cc-49db-adc8-5c84a41f088e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:52.894977 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da5539-68cc-49db-adc8-5c84a41f088e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:52.895324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da5539-68cc-49db-adc8-5c84a41f088e None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:52.896620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da5539-68cc-49db-adc8-5c84a41f088e None admin] RBAC: Authorizing `identity:get_project(project_id=e5b44df13345468b90906e747c09a977)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:52.898225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38da5539-68cc-49db-adc8-5c84a41f088e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:52.900082 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 161/311] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:14:52 2022] GET /identity/v3/projects/e5b44df13345468b90906e747c09a977 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:52.900082 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da5539-68cc-49db-adc8-5c84a41f088e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:52.912651 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-37b56547-8eb6-410e-a13e-af7e4c8d3257 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:52.914209 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-37b56547-8eb6-410e-a13e-af7e4c8d3257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:52.995218 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 151/312] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:52.996681 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b56547-8eb6-410e-a13e-af7e4c8d3257 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:53.022303 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da5539-68cc-49db-adc8-5c84a41f088e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:53.062739 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4d3d3-de4e-4542-b00a-bfbeb68e6960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:53.062739 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4d3d3-de4e-4542-b00a-bfbeb68e6960 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:53.064015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4d3d3-de4e-4542-b00a-bfbeb68e6960 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:53.064015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc4d3d3-de4e-4542-b00a-bfbeb68e6960 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:53.064504 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc4d3d3-de4e-4542-b00a-bfbeb68e6960 None admin] RBAC: Adding query filter params (role.id=d2fa7787cf3f461589be9f37afe9de6e, user.id=8e51e4ebc9d2480b94753e3b45464112, scope.system=all) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:53.064889 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc4d3d3-de4e-4542-b00a-bfbeb68e6960 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:53.066469 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc4d3d3-de4e-4542-b00a-bfbeb68e6960 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:53.073189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 162/313] 10.222.0.39 () {56 vars in 1308 bytes} [Thu Jul 28 22:14:52 2022] GET /identity/v3/role_assignments?user.id=8e51e4ebc9d2480b94753e3b45464112&scope.system=all&role.id=d2fa7787cf3f461589be9f37afe9de6e => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:53.074751 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4d3d3-de4e-4542-b00a-bfbeb68e6960 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:53.123917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:53.140729 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b56547-8eb6-410e-a13e-af7e4c8d3257 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:53.158298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:53.184013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b6b3d-0dfe-4edc-b21f-f7cbc6d7b3bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:53.184721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b6b3d-0dfe-4edc-b21f-f7cbc6d7b3bc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:53.185009 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b6b3d-0dfe-4edc-b21f-f7cbc6d7b3bc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:53.185281 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709b6b3d-0dfe-4edc-b21f-f7cbc6d7b3bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:53.186139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b6b3d-0dfe-4edc-b21f-f7cbc6d7b3bc None admin] RBAC: Adding query filter params (role.id=d2fa7787cf3f461589be9f37afe9de6e, scope.project.id=e5b44df13345468b90906e747c09a977, user.id=ccda1ef2f6c745c2b5483d6217503a54) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:53.186449 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b6b3d-0dfe-4edc-b21f-f7cbc6d7b3bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:53.187943 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709b6b3d-0dfe-4edc-b21f-f7cbc6d7b3bc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:53.195185 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 152/314] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:14:52 2022] GET /identity/v3/role_assignments?user.id=ccda1ef2f6c745c2b5483d6217503a54&scope.project.id=e5b44df13345468b90906e747c09a977&role.id=d2fa7787cf3f461589be9f37afe9de6e => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:53.211725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc4d3d3-de4e-4542-b00a-bfbeb68e6960 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:53.253693 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2004ba-4278-4273-aa73-d8b0a67b3cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:53.254287 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2004ba-4278-4273-aa73-d8b0a67b3cf1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:53.254577 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2004ba-4278-4273-aa73-d8b0a67b3cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:53.254853 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2004ba-4278-4273-aa73-d8b0a67b3cf1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:53.270101 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2004ba-4278-4273-aa73-d8b0a67b3cf1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:53.276269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-4e2004ba-4278-4273-aa73-d8b0a67b3cf1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:53.276269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:53.276269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:53.276269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:53.276269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:53.276269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:53.276269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:53.276269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:53.276269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:53.276269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:53.276269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:53.276269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:53.276269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:53.276269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:53.276269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:53.276269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:53.277811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:53.277811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:53.277811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:53.277811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:53.277811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:53.277811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:53.277811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:53.277811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:53.277811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:53.277811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:53.277811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:53.277811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:53.277811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:53.277811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:53.277811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:53.277811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:53.278624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:53.278624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:53.278624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:53.278624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:53.278624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:53.278624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:53.278624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:53.278624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:53.278624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:53.278624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:53.278624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:53.278624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:14:53.281681 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 163/315] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:53.287839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b6b3d-0dfe-4edc-b21f-f7cbc6d7b3bc None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:53.405120 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709b6b3d-0dfe-4edc-b21f-f7cbc6d7b3bc None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:53.443174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3f05c-140b-4da7-a992-8e227c781a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:53.444162 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3f05c-140b-4da7-a992-8e227c781a60 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:53.444617 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3f05c-140b-4da7-a992-8e227c781a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:53.445062 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c3f05c-140b-4da7-a992-8e227c781a60 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:53.446001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3f05c-140b-4da7-a992-8e227c781a60 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:53.446448 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3f05c-140b-4da7-a992-8e227c781a60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:53.447979 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c3f05c-140b-4da7-a992-8e227c781a60 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:53.458253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 153/316] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:53.464050 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2004ba-4278-4273-aa73-d8b0a67b3cf1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:53.571039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2004ba-4278-4273-aa73-d8b0a67b3cf1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:53.585620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:53.608656 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2a349-855f-48c6-a4c8-f58c72e5e989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:53.609309 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2a349-855f-48c6-a4c8-f58c72e5e989 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:53.609678 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2a349-855f-48c6-a4c8-f58c72e5e989 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:53.609958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2a349-855f-48c6-a4c8-f58c72e5e989 None admin] PATH_INFO: `/v3/users/2e920774520346ef96f19617348dcc3b` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:53.611159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2a349-855f-48c6-a4c8-f58c72e5e989 None admin] RBAC: Authorizing `identity:get_user(user_id=2e920774520346ef96f19617348dcc3b)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:53.612619 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2a349-855f-48c6-a4c8-f58c72e5e989 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:53.618973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 164/317] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:53 2022] GET /identity/v3/users/2e920774520346ef96f19617348dcc3b => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:53.619750 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3f05c-140b-4da7-a992-8e227c781a60 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:53.637704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:53.728305 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c3f05c-140b-4da7-a992-8e227c781a60 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:53.764139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3ffbe-f3b3-4731-8f3f-c17134132ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:53.764766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3ffbe-f3b3-4731-8f3f-c17134132ad4 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:53.765358 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3ffbe-f3b3-4731-8f3f-c17134132ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:53.765716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3ffbe-f3b3-4731-8f3f-c17134132ad4 None admin] PATH_INFO: `/v3/system/users/2e920774520346ef96f19617348dcc3b/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:53.767410 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3ffbe-f3b3-4731-8f3f-c17134132ad4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2e920774520346ef96f19617348dcc3b, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:53.768981 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3ffbe-f3b3-4731-8f3f-c17134132ad4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:53.778304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 154/318] 10.222.0.39 () {58 vars in 1334 bytes} [Thu Jul 28 22:14:53 2022] PUT /identity/v3/system/users/2e920774520346ef96f19617348dcc3b/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:14:54.972128 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cceee33-ebbc-4e65-b9e9-a9b3d2b2d913 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:54.972653 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cceee33-ebbc-4e65-b9e9-a9b3d2b2d913 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:54.972653 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cceee33-ebbc-4e65-b9e9-a9b3d2b2d913 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:54.974214 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 165/319] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:54.979044 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0f377-1789-48b3-ae21-921836d9aedd None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:54.979438 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0f377-1789-48b3-ae21-921836d9aedd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:54.980592 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0f377-1789-48b3-ae21-921836d9aedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:54.998863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-15f0f377-1789-48b3-ae21-921836d9aedd None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:55.000043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-15f0f377-1789-48b3-ae21-921836d9aedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:55.068735 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 155/320] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:55.090475 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cceee33-ebbc-4e65-b9e9-a9b3d2b2d913 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:55.156018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81914fa0-465e-4940-8a30-bed85a631780 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:55.156426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81914fa0-465e-4940-8a30-bed85a631780 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:55.156608 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81914fa0-465e-4940-8a30-bed85a631780 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:55.157981 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 156/321] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:55.163058 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb367d5-4686-41f1-bd69-62e0b28c9680 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:55.163339 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb367d5-4686-41f1-bd69-62e0b28c9680 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:55.163611 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb367d5-4686-41f1-bd69-62e0b28c9680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:55.182053 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-dcb367d5-4686-41f1-bd69-62e0b28c9680 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:55.184081 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-dcb367d5-4686-41f1-bd69-62e0b28c9680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:55.186181 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:55.217246 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cceee33-ebbc-4e65-b9e9-a9b3d2b2d913 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:55.253322 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db260d73-ee0a-42a5-941a-5d0b3cd1eba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:55.253906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db260d73-ee0a-42a5-941a-5d0b3cd1eba3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:55.254182 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db260d73-ee0a-42a5-941a-5d0b3cd1eba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:55.254454 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db260d73-ee0a-42a5-941a-5d0b3cd1eba3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:55.255578 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db260d73-ee0a-42a5-941a-5d0b3cd1eba3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:55.257318 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db260d73-ee0a-42a5-941a-5d0b3cd1eba3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:55.257433 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 157/322] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:14:55.259030 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 166/323] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:14:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:55.267163 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb367d5-4686-41f1-bd69-62e0b28c9680 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:55.279286 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db260d73-ee0a-42a5-941a-5d0b3cd1eba3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:55.336985 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:55.372479 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb367d5-4686-41f1-bd69-62e0b28c9680 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:55.391526 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db260d73-ee0a-42a5-941a-5d0b3cd1eba3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:55.407859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbe4b7-7041-49ea-92bb-9f36f6508c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:55.408478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbe4b7-7041-49ea-92bb-9f36f6508c4b None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:55.408759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbe4b7-7041-49ea-92bb-9f36f6508c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:55.409034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bbe4b7-7041-49ea-92bb-9f36f6508c4b None admin] PATH_INFO: `/v3/groups` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:55.409762 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbe4b7-7041-49ea-92bb-9f36f6508c4b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:55.411210 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bbe4b7-7041-49ea-92bb-9f36f6508c4b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:55.422802 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:55.428921 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd8c5a-c27c-4a32-9228-14828f6a6493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:55.429801 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd8c5a-c27c-4a32-9228-14828f6a6493 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:55.430002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd8c5a-c27c-4a32-9228-14828f6a6493 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:55.430308 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cd8c5a-c27c-4a32-9228-14828f6a6493 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:55.437116 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 158/324] 10.222.0.39 () {60 vars in 1134 bytes} [Thu Jul 28 22:14:55 2022] POST /identity/v3/groups => generated 226 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:14:55.447806 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cd8c5a-c27c-4a32-9228-14828f6a6493 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:55.454066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-72cd8c5a-c27c-4a32-9228-14828f6a6493 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 22:14:55.454066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:55.454066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:55.454066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:55.454066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:55.454066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:55.454066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:55.454066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:55.454066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:55.454066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:55.454066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:55.454066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:55.454066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:55.454066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:55.454066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:55.454066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:55.455031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:55.455031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:55.455031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:55.455031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:55.455031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:55.455031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:55.455031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:55.455031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:55.455031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:55.455031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:55.455031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:55.455031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:55.455031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:55.455031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:55.455031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:55.455031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:55.455843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:55.455843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:55.455843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:55.455843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:55.455843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:55.455843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:55.455843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:55.455843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:55.455843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:55.455843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:55.455843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 22:14:55.455843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:14:55.457095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 167/325] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:14:55.461851 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbe4b7-7041-49ea-92bb-9f36f6508c4b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:55.576631 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bbe4b7-7041-49ea-92bb-9f36f6508c4b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:55.614740 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5fa687-5178-4550-9b31-b6c7aee819fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:55.615364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fa687-5178-4550-9b31-b6c7aee819fb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:55.615643 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fa687-5178-4550-9b31-b6c7aee819fb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:55.615913 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fa687-5178-4550-9b31-b6c7aee819fb None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:55.616633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5fa687-5178-4550-9b31-b6c7aee819fb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:55.616936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5fa687-5178-4550-9b31-b6c7aee819fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:55.618626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5fa687-5178-4550-9b31-b6c7aee819fb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:55.627808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 159/326] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:55.633225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd8c5a-c27c-4a32-9228-14828f6a6493 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:55.750122 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cd8c5a-c27c-4a32-9228-14828f6a6493 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:55.761145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:55.786055 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ccf01-1a56-408d-be44-5879d7861a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:55.786791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ccf01-1a56-408d-be44-5879d7861a75 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:55.787073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ccf01-1a56-408d-be44-5879d7861a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:55.787479 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918ccf01-1a56-408d-be44-5879d7861a75 None admin] PATH_INFO: `/v3/users/8e51e4ebc9d2480b94753e3b45464112` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:55.788387 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a4448-c6b4-4a38-946d-8d3e034c8000 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:55.788576 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ccf01-1a56-408d-be44-5879d7861a75 None admin] RBAC: Authorizing `identity:get_user(user_id=8e51e4ebc9d2480b94753e3b45464112)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:55.788671 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a4448-c6b4-4a38-946d-8d3e034c8000 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:55.788940 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a4448-c6b4-4a38-946d-8d3e034c8000 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:55.790025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918ccf01-1a56-408d-be44-5879d7861a75 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:55.790482 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 160/327] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:55.792185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 168/328] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:55 2022] GET /identity/v3/users/8e51e4ebc9d2480b94753e3b45464112 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:55.796477 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eec74e-2ea0-4d14-87a4-bd823000782d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:55.796757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eec74e-2ea0-4d14-87a4-bd823000782d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:55.797027 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eec74e-2ea0-4d14-87a4-bd823000782d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:55.798814 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ccf01-1a56-408d-be44-5879d7861a75 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:55.826965 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f3eec74e-2ea0-4d14-87a4-bd823000782d None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:55.828113 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f3eec74e-2ea0-4d14-87a4-bd823000782d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:55.903320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:55.921990 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918ccf01-1a56-408d-be44-5879d7861a75 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:55.923003 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 161/329] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:14:55.945803 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eec74e-2ea0-4d14-87a4-bd823000782d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:55.961283 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad45688-9d57-486a-bd1f-08758b897472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:55.962014 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad45688-9d57-486a-bd1f-08758b897472 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:55.962296 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad45688-9d57-486a-bd1f-08758b897472 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:55.962567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad45688-9d57-486a-bd1f-08758b897472 None admin] PATH_INFO: `/v3/system/users/8e51e4ebc9d2480b94753e3b45464112/roles/d2fa7787cf3f461589be9f37afe9de6e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:55.964488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad45688-9d57-486a-bd1f-08758b897472 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8e51e4ebc9d2480b94753e3b45464112, role_id=d2fa7787cf3f461589be9f37afe9de6e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:55.965736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad45688-9d57-486a-bd1f-08758b897472 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:55.978936 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 169/330] 10.222.0.39 () {58 vars in 1334 bytes} [Thu Jul 28 22:14:55 2022] PUT /identity/v3/system/users/8e51e4ebc9d2480b94753e3b45464112/roles/d2fa7787cf3f461589be9f37afe9de6e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:14:56.081563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eec74e-2ea0-4d14-87a4-bd823000782d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:56.122700 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a0e4b-7368-4d18-bf1f-c89dcb3c3d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:56.123341 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a0e4b-7368-4d18-bf1f-c89dcb3c3d19 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:56.123630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a0e4b-7368-4d18-bf1f-c89dcb3c3d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:56.123907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a0e4b-7368-4d18-bf1f-c89dcb3c3d19 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:56.137274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a0e4b-7368-4d18-bf1f-c89dcb3c3d19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:56.144568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-380a0e4b-7368-4d18-bf1f-c89dcb3c3d19 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:56.144568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:56.144568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:56.144568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:56.144568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:56.144568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:56.144568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:56.144568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:56.144568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:56.144568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:56.144568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:56.144568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:56.144568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:56.144568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:56.144568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:56.144568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:56.146039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:56.146039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:56.146039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:56.146039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:56.146039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:56.146039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:56.146039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:56.146039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:56.146039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:56.146039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:56.146039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:56.146039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:56.146039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:56.146039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:56.146039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:56.146039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:56.147969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:56.147969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:56.147969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:56.147969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:56.147969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:56.147969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:56.147969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:56.147969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:56.147969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:56.147969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:56.147969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:56.147969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:56.147969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 162/331] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:14:56.153567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad45688-9d57-486a-bd1f-08758b897472 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:56.222184 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:56.302505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad45688-9d57-486a-bd1f-08758b897472 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:56.345458 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd6eb4-e3f7-4300-a701-bcf0532a24d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:56.346132 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd6eb4-e3f7-4300-a701-bcf0532a24d8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:56.346412 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd6eb4-e3f7-4300-a701-bcf0532a24d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:56.346684 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd6eb4-e3f7-4300-a701-bcf0532a24d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:56.347421 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd6eb4-e3f7-4300-a701-bcf0532a24d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:56.347729 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd6eb4-e3f7-4300-a701-bcf0532a24d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:56.349116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd6eb4-e3f7-4300-a701-bcf0532a24d8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:56.360539 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 170/332] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:56.366008 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a0e4b-7368-4d18-bf1f-c89dcb3c3d19 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:56.423936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:56.483795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a0e4b-7368-4d18-bf1f-c89dcb3c3d19 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:56.521288 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf17b01-03d4-43e0-82da-539a92774804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:56.521970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf17b01-03d4-43e0-82da-539a92774804 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:56.522254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf17b01-03d4-43e0-82da-539a92774804 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:56.522529 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf17b01-03d4-43e0-82da-539a92774804 None admin] PATH_INFO: `/v3/users/2e920774520346ef96f19617348dcc3b` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:56.523708 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf17b01-03d4-43e0-82da-539a92774804 None admin] RBAC: Authorizing `identity:get_user(user_id=2e920774520346ef96f19617348dcc3b)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:56.525169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf17b01-03d4-43e0-82da-539a92774804 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:56.528092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 163/333] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:56 2022] GET /identity/v3/users/2e920774520346ef96f19617348dcc3b => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:14:56.534627 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd6eb4-e3f7-4300-a701-bcf0532a24d8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:56.741638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd6eb4-e3f7-4300-a701-bcf0532a24d8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:56.746569 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:56.819462 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef344b63-ba63-411e-9a49-c0f00e996e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:56.820440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef344b63-ba63-411e-9a49-c0f00e996e47 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:56.820925 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef344b63-ba63-411e-9a49-c0f00e996e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:56.821398 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef344b63-ba63-411e-9a49-c0f00e996e47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:56.822428 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef344b63-ba63-411e-9a49-c0f00e996e47 None admin] RBAC: Adding query filter params (role.id=4912a35790fe4d4297540156083d81d1, user.id=2e920774520346ef96f19617348dcc3b, scope.system=all) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:56.823048 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef344b63-ba63-411e-9a49-c0f00e996e47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:56.825247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef344b63-ba63-411e-9a49-c0f00e996e47 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:56.845211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 171/334] 10.222.0.39 () {56 vars in 1308 bytes} [Thu Jul 28 22:14:56 2022] GET /identity/v3/role_assignments?user.id=2e920774520346ef96f19617348dcc3b&scope.system=all&role.id=4912a35790fe4d4297540156083d81d1 => generated 510 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:58.630620 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d41d91-a0f8-4e2c-aaa9-7db28fb044bd None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:58.631127 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d41d91-a0f8-4e2c-aaa9-7db28fb044bd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:58.631203 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d41d91-a0f8-4e2c-aaa9-7db28fb044bd None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:58.632765 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 164/335] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:58.647046 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace99924-4326-4a48-9c4d-6e0c37145cfb None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:58.647956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace99924-4326-4a48-9c4d-6e0c37145cfb None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:58.648511 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace99924-4326-4a48-9c4d-6e0c37145cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:58.654114 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c799d24-7971-41b9-b9a6-d5ffbf14dec2 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:58.654667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c799d24-7971-41b9-b9a6-d5ffbf14dec2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:58.655134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c799d24-7971-41b9-b9a6-d5ffbf14dec2 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:58.656911 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 165/336] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:58.663782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25fd7d-8e5d-4675-bc6d-5b32520cc0c0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:58.664327 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25fd7d-8e5d-4675-bc6d-5b32520cc0c0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:58.664794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25fd7d-8e5d-4675-bc6d-5b32520cc0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:58.674520 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ace99924-4326-4a48-9c4d-6e0c37145cfb None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:58.675792 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ace99924-4326-4a48-9c4d-6e0c37145cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:58.696720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1a25fd7d-8e5d-4675-bc6d-5b32520cc0c0 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:58.701542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1a25fd7d-8e5d-4675-bc6d-5b32520cc0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:58.777872 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 166/337] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:14:58.779553 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96851b4-3636-41ea-b10e-94de00817c2e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:58.780003 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96851b4-3636-41ea-b10e-94de00817c2e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:58.780450 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96851b4-3636-41ea-b10e-94de00817c2e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:58.782129 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 167/338] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:58.791753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 172/339] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:58.804219 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace99924-4326-4a48-9c4d-6e0c37145cfb None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:58.804639 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538bc036-7be5-4e27-95dc-21e68c0c1acf None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:58.804965 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538bc036-7be5-4e27-95dc-21e68c0c1acf None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:58.805244 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538bc036-7be5-4e27-95dc-21e68c0c1acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:58.825098 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-538bc036-7be5-4e27-95dc-21e68c0c1acf None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:58.826355 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-538bc036-7be5-4e27-95dc-21e68c0c1acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:58.878169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:58.912980 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 168/340] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:58.914100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538bc036-7be5-4e27-95dc-21e68c0c1acf None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:58.942563 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:58.962805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace99924-4326-4a48-9c4d-6e0c37145cfb None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:59.013106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981895e-75bd-42a9-8a79-6b8b65fa539b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:59.013799 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981895e-75bd-42a9-8a79-6b8b65fa539b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:59.014083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981895e-75bd-42a9-8a79-6b8b65fa539b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:59.014359 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981895e-75bd-42a9-8a79-6b8b65fa539b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:59.029499 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5981895e-75bd-42a9-8a79-6b8b65fa539b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:59.036304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-5981895e-75bd-42a9-8a79-6b8b65fa539b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:59.036304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:59.036304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:59.036304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:59.036304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:59.036304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:59.036304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:59.036304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:59.036304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:59.036304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:59.036304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:59.036304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:59.036304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:59.036304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:59.036304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:59.036304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:59.037395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:59.037395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:59.037395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:59.037395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:59.037395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:59.037395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:59.037395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:59.037395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:59.037395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:59.037395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:59.037395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:59.037395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:59.037395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:59.037395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:59.037395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:59.037395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:59.038247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:59.038247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:59.038247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:59.038247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:59.038247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:59.038247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:59.038247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:59.038247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:59.038247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:59.038247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:59.038247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:59.038247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:14:59.038247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 173/341] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:59.038918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83825e8a-1f3e-4507-b0a2-548580b535bc None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:59.038918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83825e8a-1f3e-4507-b0a2-548580b535bc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:59.039279 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83825e8a-1f3e-4507-b0a2-548580b535bc None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:59.041005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 174/342] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:14:59.042649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83825e8a-1f3e-4507-b0a2-548580b535bc None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:59.051877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538bc036-7be5-4e27-95dc-21e68c0c1acf None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:59.095713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c525ce4-ecb8-43fb-b03e-63320fdb5d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:59.096372 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c525ce4-ecb8-43fb-b03e-63320fdb5d5c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:59.096652 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c525ce4-ecb8-43fb-b03e-63320fdb5d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:59.096929 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c525ce4-ecb8-43fb-b03e-63320fdb5d5c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:59.111419 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c525ce4-ecb8-43fb-b03e-63320fdb5d5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:59.117903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-9c525ce4-ecb8-43fb-b03e-63320fdb5d5c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 22:14:59.117903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:59.117903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:59.117903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:59.117903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:59.117903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:59.117903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:59.117903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:59.117903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:59.117903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:59.117903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:59.117903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:59.117903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:59.117903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:59.117903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:59.117903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:59.118783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:59.118783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:59.118783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:59.118783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:59.118783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:59.118783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:59.118783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:59.118783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:59.118783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:59.118783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:59.118783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:59.118783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:59.118783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:59.118783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:59.118783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:59.118783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:59.119591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:59.119591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:59.119591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:59.119591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:59.119591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:59.119591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:59.119591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:59.119591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:59.119591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:59.119591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:59.119591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 28 22:14:59.119591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:14:59.119591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 169/343] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:59.120922 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34cb3c-61cc-49e7-8a0e-733ccf7b90e2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:59.120922 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34cb3c-61cc-49e7-8a0e-733ccf7b90e2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:59.121595 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db34cb3c-61cc-49e7-8a0e-733ccf7b90e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:59.144570 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-db34cb3c-61cc-49e7-8a0e-733ccf7b90e2 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:14:59.146213 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-db34cb3c-61cc-49e7-8a0e-733ccf7b90e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:14:59.187446 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:59.192940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83825e8a-1f3e-4507-b0a2-548580b535bc None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:59.234066 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 170/344] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:14:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:14:59.234976 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34cb3c-61cc-49e7-8a0e-733ccf7b90e2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:59.237698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc4bec-a8c5-4dd7-94c6-a1f17fff656c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:59.239455 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc4bec-a8c5-4dd7-94c6-a1f17fff656c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:59.239949 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc4bec-a8c5-4dd7-94c6-a1f17fff656c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:59.240428 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc4bec-a8c5-4dd7-94c6-a1f17fff656c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:59.253128 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc4bec-a8c5-4dd7-94c6-a1f17fff656c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:59.259588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-34fc4bec-a8c5-4dd7-94c6-a1f17fff656c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:14:59.259588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:59.259588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:59.259588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:59.259588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:59.259588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:59.259588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:59.259588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:59.259588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:59.259588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:59.259588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:59.259588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:59.259588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:59.259588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:59.259588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:59.259588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:59.260667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:59.260667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:59.260667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:59.260667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:59.260667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:59.260667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:59.260667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:59.260667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:59.260667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:59.260667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:59.260667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:59.260667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:59.260667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:59.260667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:59.260667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:59.260667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:59.261487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:59.261487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:59.261487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:59.261487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:59.261487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:59.261487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:59.261487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:59.261487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:59.261487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:59.261487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:59.261487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:14:59.261487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:14:59.264190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 175/345] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:14:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:59.265753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc4bec-a8c5-4dd7-94c6-a1f17fff656c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:59.287526 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:59.372154 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db34cb3c-61cc-49e7-8a0e-733ccf7b90e2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:59.405562 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc4bec-a8c5-4dd7-94c6-a1f17fff656c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:59.417207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5addb2-d53a-4ed8-9562-ac8d2ee01f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:59.417678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5addb2-d53a-4ed8-9562-ac8d2ee01f2e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:59.417964 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5addb2-d53a-4ed8-9562-ac8d2ee01f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:59.418657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5addb2-d53a-4ed8-9562-ac8d2ee01f2e None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:59.419681 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5addb2-d53a-4ed8-9562-ac8d2ee01f2e None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:59.419993 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5addb2-d53a-4ed8-9562-ac8d2ee01f2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:59.421680 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5addb2-d53a-4ed8-9562-ac8d2ee01f2e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:59.435381 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 171/346] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:59.436911 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5addb2-d53a-4ed8-9562-ac8d2ee01f2e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:59.458538 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6d45e-eefd-421b-a0c4-2034442d7312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:59.459101 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6d45e-eefd-421b-a0c4-2034442d7312 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:59.459384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6d45e-eefd-421b-a0c4-2034442d7312 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:59.459658 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6d45e-eefd-421b-a0c4-2034442d7312 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:59.460546 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6d45e-eefd-421b-a0c4-2034442d7312 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:59.460853 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6d45e-eefd-421b-a0c4-2034442d7312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:59.462479 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6d45e-eefd-421b-a0c4-2034442d7312 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:59.473242 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 176/347] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:14:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:59.475629 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6d45e-eefd-421b-a0c4-2034442d7312 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:59.490903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:59.577380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:59.580845 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5addb2-d53a-4ed8-9562-ac8d2ee01f2e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:59.613117 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6d45e-eefd-421b-a0c4-2034442d7312 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:59.630981 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c04efa-7d1e-42b8-a59e-05eb4229ad84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:59.632556 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c04efa-7d1e-42b8-a59e-05eb4229ad84 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:59.633050 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c04efa-7d1e-42b8-a59e-05eb4229ad84 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:59.633518 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c04efa-7d1e-42b8-a59e-05eb4229ad84 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:59.634464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c04efa-7d1e-42b8-a59e-05eb4229ad84 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:14:59.635339 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c04efa-7d1e-42b8-a59e-05eb4229ad84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:59.638297 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c04efa-7d1e-42b8-a59e-05eb4229ad84 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:59.649941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 172/348] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:14:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:59.650961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c04efa-7d1e-42b8-a59e-05eb4229ad84 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:59.655445 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ba702-98e4-458c-819a-f4ec9a22242e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:59.656689 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ba702-98e4-458c-819a-f4ec9a22242e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:59.657124 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ba702-98e4-458c-819a-f4ec9a22242e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:59.657552 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0ba702-98e4-458c-819a-f4ec9a22242e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:59.671123 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0ba702-98e4-458c-819a-f4ec9a22242e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:14:59.676948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-be0ba702-98e4-458c-819a-f4ec9a22242e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:59.676948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:14:59.676948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:14:59.676948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:14:59.676948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:14:59.676948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:14:59.676948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:14:59.676948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:14:59.676948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:14:59.676948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:14:59.676948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:14:59.676948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:14:59.676948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:14:59.676948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:14:59.676948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:14:59.676948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:14:59.678624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:14:59.678624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:14:59.678624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:14:59.678624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:14:59.678624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:14:59.678624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:14:59.678624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:14:59.678624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:14:59.678624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:14:59.678624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:14:59.678624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:14:59.678624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:14:59.678624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:14:59.678624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:14:59.678624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:14:59.678624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:14:59.679414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:14:59.679414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:14:59.679414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:14:59.679414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:14:59.679414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:14:59.679414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:14:59.679414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:14:59.679414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:14:59.679414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:14:59.679414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:14:59.679414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:14:59.679414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:14:59.685652 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 177/349] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:14:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:14:59.686756 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ba702-98e4-458c-819a-f4ec9a22242e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:59.790392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:14:59.810005 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c04efa-7d1e-42b8-a59e-05eb4229ad84 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:59.846600 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0ba702-98e4-458c-819a-f4ec9a22242e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:14:59.877259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e1a6f-3aba-483d-ac66-0cccaea180b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:59.878859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e1a6f-3aba-483d-ac66-0cccaea180b6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:59.879584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e1a6f-3aba-483d-ac66-0cccaea180b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:59.880374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8e1a6f-3aba-483d-ac66-0cccaea180b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:59.912921 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee653d-12dd-4917-aa2d-c63f28800cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:14:59.918501 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee653d-12dd-4917-aa2d-c63f28800cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:14:59.918501 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee653d-12dd-4917-aa2d-c63f28800cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:14:59.918501 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ee653d-12dd-4917-aa2d-c63f28800cc8 None admin] PATH_INFO: `/v3/users/8e51e4ebc9d2480b94753e3b45464112` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:14:59.919045 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee653d-12dd-4917-aa2d-c63f28800cc8 None admin] RBAC: Authorizing `identity:get_user(user_id=8e51e4ebc9d2480b94753e3b45464112)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:14:59.921965 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ee653d-12dd-4917-aa2d-c63f28800cc8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:14:59.929554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 178/350] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:14:59 2022] GET /identity/v3/users/8e51e4ebc9d2480b94753e3b45464112 => generated 293 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:14:59.929554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee653d-12dd-4917-aa2d-c63f28800cc8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:14:59.931207 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:00.001828 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e1a6f-3aba-483d-ac66-0cccaea180b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:00.003291 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8e1a6f-3aba-483d-ac66-0cccaea180b6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:00.092483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ee653d-12dd-4917-aa2d-c63f28800cc8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:00.138890 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f63d2-b969-4bb4-a260-cbd801cc9e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:00.139516 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f63d2-b969-4bb4-a260-cbd801cc9e81 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:00.140045 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f63d2-b969-4bb4-a260-cbd801cc9e81 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:00.140319 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f63d2-b969-4bb4-a260-cbd801cc9e81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:00.147893 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:00.160721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 173/351] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:15:00.161304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e1a6f-3aba-483d-ac66-0cccaea180b6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:00.242819 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f63d2-b969-4bb4-a260-cbd801cc9e81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:00.244528 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f63d2-b969-4bb4-a260-cbd801cc9e81 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:00.247588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:00.299362 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8e1a6f-3aba-483d-ac66-0cccaea180b6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:00.336233 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36ea2c-4363-4929-a7bc-648fa4fefc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:00.336855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36ea2c-4363-4929-a7bc-648fa4fefc41 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:00.337143 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36ea2c-4363-4929-a7bc-648fa4fefc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:00.337438 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f36ea2c-4363-4929-a7bc-648fa4fefc41 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:00.338187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36ea2c-4363-4929-a7bc-648fa4fefc41 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:00.338493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36ea2c-4363-4929-a7bc-648fa4fefc41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:00.340087 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f36ea2c-4363-4929-a7bc-648fa4fefc41 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:00.350294 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 174/352] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:15:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:00.351131 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36ea2c-4363-4929-a7bc-648fa4fefc41 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:00.384756 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 179/353] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:14:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:15:00.385839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9e12a-62ab-433b-a19f-40066cf7a397 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:00.386216 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9e12a-62ab-433b-a19f-40066cf7a397 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:00.386495 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9e12a-62ab-433b-a19f-40066cf7a397 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:00.387907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 180/354] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:00.388482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9e12a-62ab-433b-a19f-40066cf7a397 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:00.421335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:00.476582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f36ea2c-4363-4929-a7bc-648fa4fefc41 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:00.503305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9e12a-62ab-433b-a19f-40066cf7a397 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:00.504849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:00.520345 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7962d0f7-ae73-4e1c-bbee-fdc6489c37f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:00.521342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962d0f7-ae73-4e1c-bbee-fdc6489c37f5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:00.521665 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962d0f7-ae73-4e1c-bbee-fdc6489c37f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:00.521946 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7962d0f7-ae73-4e1c-bbee-fdc6489c37f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:00.522881 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962d0f7-ae73-4e1c-bbee-fdc6489c37f5 None admin] RBAC: Adding query filter params (role.id=d2fa7787cf3f461589be9f37afe9de6e, user.id=8e51e4ebc9d2480b94753e3b45464112, scope.system=all) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:00.523191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962d0f7-ae73-4e1c-bbee-fdc6489c37f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:00.524648 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7962d0f7-ae73-4e1c-bbee-fdc6489c37f5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:00.539204 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 175/355] 10.222.0.39 () {56 vars in 1308 bytes} [Thu Jul 28 22:15:00 2022] GET /identity/v3/role_assignments?user.id=8e51e4ebc9d2480b94753e3b45464112&scope.system=all&role.id=d2fa7787cf3f461589be9f37afe9de6e => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:00.539204 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead24c78-6038-48c7-949d-48d5291047f5 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:00.539433 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead24c78-6038-48c7-949d-48d5291047f5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:00.539501 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead24c78-6038-48c7-949d-48d5291047f5 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:00.541073 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 176/356] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:00.543910 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead24c78-6038-48c7-949d-48d5291047f5 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:00.550895 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644002ee-513f-4aa8-a3fe-862ebe9ec449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:00.551573 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644002ee-513f-4aa8-a3fe-862ebe9ec449 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:00.551860 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644002ee-513f-4aa8-a3fe-862ebe9ec449 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:00.552136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644002ee-513f-4aa8-a3fe-862ebe9ec449 None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:00.553391 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644002ee-513f-4aa8-a3fe-862ebe9ec449 None admin] RBAC: Authorizing `identity:get_project(project_id=e5b44df13345468b90906e747c09a977)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:00.555016 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644002ee-513f-4aa8-a3fe-862ebe9ec449 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:00.556972 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 181/357] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:15:00 2022] GET /identity/v3/projects/e5b44df13345468b90906e747c09a977 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:00.560236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644002ee-513f-4aa8-a3fe-862ebe9ec449 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:00.698882 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644002ee-513f-4aa8-a3fe-862ebe9ec449 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:00.702897 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead24c78-6038-48c7-949d-48d5291047f5 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:00.720923 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:00.753505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc49ebc-315f-4d8d-8a2b-da8a59842bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:00.754736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc49ebc-315f-4d8d-8a2b-da8a59842bfa None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:00.755289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc49ebc-315f-4d8d-8a2b-da8a59842bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:00.756046 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc49ebc-315f-4d8d-8a2b-da8a59842bfa None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:00.757630 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc49ebc-315f-4d8d-8a2b-da8a59842bfa None admin] RBAC: Authorizing `identity:get_project(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:00.759994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc49ebc-315f-4d8d-8a2b-da8a59842bfa None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:00.762920 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98066121-020f-4946-8eb0-2ce847ea6756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:00.763772 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98066121-020f-4946-8eb0-2ce847ea6756 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:00.764055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98066121-020f-4946-8eb0-2ce847ea6756 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:00.764327 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98066121-020f-4946-8eb0-2ce847ea6756 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:00.765621 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 182/358] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:15:00 2022] GET /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:00.767262 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc76db-cb39-489b-8848-97d270b43703 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:00.770802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc76db-cb39-489b-8848-97d270b43703 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:00.772595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc76db-cb39-489b-8848-97d270b43703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:00.803305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-bbdc76db-cb39-489b-8848-97d270b43703 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:00.810653 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bbdc76db-cb39-489b-8848-97d270b43703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:00.853494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:00.868784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98066121-020f-4946-8eb0-2ce847ea6756 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:00.871190 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98066121-020f-4946-8eb0-2ce847ea6756 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:00.908757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 183/359] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:00.910281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a486393-30c3-4295-8517-7c4ebe1e99c8 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:00.910568 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a486393-30c3-4295-8517-7c4ebe1e99c8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:00.910909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a486393-30c3-4295-8517-7c4ebe1e99c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:00.929640 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1a486393-30c3-4295-8517-7c4ebe1e99c8 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:00.930815 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1a486393-30c3-4295-8517-7c4ebe1e99c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:01.014242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 177/360] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:15:01.015851 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98066121-020f-4946-8eb0-2ce847ea6756 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:01.027275 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 184/361] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:01.028445 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a486393-30c3-4295-8517-7c4ebe1e99c8 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:01.035632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:01.159021 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:01.164375 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a486393-30c3-4295-8517-7c4ebe1e99c8 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:01.168330 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98066121-020f-4946-8eb0-2ce847ea6756 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:01.211110 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e206d70-e8c8-4a0f-9bb1-f77a2c6aea5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:01.212226 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e206d70-e8c8-4a0f-9bb1-f77a2c6aea5c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:01.212226 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e206d70-e8c8-4a0f-9bb1-f77a2c6aea5c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:01.212391 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e206d70-e8c8-4a0f-9bb1-f77a2c6aea5c None admin] PATH_INFO: `/v3/groups/62ee23836ebe49419e6178e20ed8a2ec` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:01.216069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd46c1-e875-4c6b-9579-ca1f8d3993a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:01.216746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd46c1-e875-4c6b-9579-ca1f8d3993a5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:01.217031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd46c1-e875-4c6b-9579-ca1f8d3993a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:01.217694 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd46c1-e875-4c6b-9579-ca1f8d3993a5 None admin] PATH_INFO: `/v3/groups/62ee23836ebe49419e6178e20ed8a2ec` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:01.221025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e206d70-e8c8-4a0f-9bb1-f77a2c6aea5c None admin] RBAC: Authorizing `identity:get_group(group_id=62ee23836ebe49419e6178e20ed8a2ec)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:01.223009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e206d70-e8c8-4a0f-9bb1-f77a2c6aea5c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:01.223909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd46c1-e875-4c6b-9579-ca1f8d3993a5 None admin] RBAC: Authorizing `identity:get_group(group_id=62ee23836ebe49419e6178e20ed8a2ec)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:01.225560 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 185/362] 10.222.0.39 () {56 vars in 1180 bytes} [Thu Jul 28 22:15:01 2022] GET /identity/v3/groups/62ee23836ebe49419e6178e20ed8a2ec => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:01.225667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd46c1-e875-4c6b-9579-ca1f8d3993a5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:01.227028 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e206d70-e8c8-4a0f-9bb1-f77a2c6aea5c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:01.228014 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 178/363] 10.222.0.39 () {56 vars in 1180 bytes} [Thu Jul 28 22:15:01 2022] GET /identity/v3/groups/62ee23836ebe49419e6178e20ed8a2ec => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:01.231373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd46c1-e875-4c6b-9579-ca1f8d3993a5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:01.332261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:01.365348 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd46c1-e875-4c6b-9579-ca1f8d3993a5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:01.369667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e206d70-e8c8-4a0f-9bb1-f77a2c6aea5c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:01.414428 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3002c-14a6-4ae2-b53b-249df4597dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:01.414649 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95ff08-acbc-4028-98ec-89ae5ec05f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:01.414741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3002c-14a6-4ae2-b53b-249df4597dba None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:01.414812 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95ff08-acbc-4028-98ec-89ae5ec05f3a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:01.414878 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3002c-14a6-4ae2-b53b-249df4597dba None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:01.414943 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95ff08-acbc-4028-98ec-89ae5ec05f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:01.415018 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c3002c-14a6-4ae2-b53b-249df4597dba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:01.415086 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95ff08-acbc-4028-98ec-89ae5ec05f3a None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:01.416145 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95ff08-acbc-4028-98ec-89ae5ec05f3a None admin] RBAC: Authorizing `identity:get_project(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:01.417824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95ff08-acbc-4028-98ec-89ae5ec05f3a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:01.419540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 179/364] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:15:01 2022] GET /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:01.420755 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95ff08-acbc-4028-98ec-89ae5ec05f3a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:01.434251 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c3002c-14a6-4ae2-b53b-249df4597dba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:01.441213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-84c3002c-14a6-4ae2-b53b-249df4597dba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:15:01.441213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:01.441213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:01.441213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:01.441213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:01.441213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:01.441213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:01.441213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:01.441213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:01.441213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:01.441213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:01.441213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:01.441213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:01.441213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:01.441213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:01.441213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:01.442324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:01.442324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:01.442324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:01.442324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:01.442324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:01.442324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:01.442324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:01.442324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:01.442324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:01.442324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:01.442324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:01.442324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:01.442324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:01.442324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:01.442324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:01.442324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:01.443405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:01.443405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:01.443405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:01.443405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:01.443405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:01.443405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:01.443405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:01.443405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:01.443405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:01.443405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:01.443405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:15:01.443405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:01.443405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 186/365] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:15:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:01.444479 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3002c-14a6-4ae2-b53b-249df4597dba None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:01.483422 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:01.549446 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95ff08-acbc-4028-98ec-89ae5ec05f3a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:01.592721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad337f-d7fb-41f5-ab14-a05832ebe8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:01.593298 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad337f-d7fb-41f5-ab14-a05832ebe8ef None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:01.593942 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c3002c-14a6-4ae2-b53b-249df4597dba None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:01.594457 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad337f-d7fb-41f5-ab14-a05832ebe8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:01.595303 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ad337f-d7fb-41f5-ab14-a05832ebe8ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:01.607195 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ad337f-d7fb-41f5-ab14-a05832ebe8ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:01.612613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-52ad337f-d7fb-41f5-ab14-a05832ebe8ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:15:01.612613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:01.612613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:01.612613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:01.612613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:01.612613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:01.612613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:01.612613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:01.612613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:01.612613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:01.612613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:01.612613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:01.612613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:01.612613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:01.612613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:01.612613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:01.613496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:01.613496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:01.613496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:01.613496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:01.613496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:01.613496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:01.613496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:01.613496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:01.613496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:01.613496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:01.613496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:01.613496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:01.613496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:01.613496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:01.613496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:01.613496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:01.614789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:01.614789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:01.614789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:01.614789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:01.614789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:01.614789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:01.614789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:01.614789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:01.614789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:01.614789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:01.614789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:15:01.614789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:01.614789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 180/366] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:15:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:01.615768 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad337f-d7fb-41f5-ab14-a05832ebe8ef None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:01.623369 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:01.642933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a45de-41b7-4b05-9666-dbbae8f98bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:01.643558 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a45de-41b7-4b05-9666-dbbae8f98bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:01.644638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a45de-41b7-4b05-9666-dbbae8f98bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:01.644917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36a45de-41b7-4b05-9666-dbbae8f98bf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:01.645771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a45de-41b7-4b05-9666-dbbae8f98bf1 None admin] RBAC: Adding query filter params (group.id=62ee23836ebe49419e6178e20ed8a2ec, role.id=4146966c23674402aebb9a5b21cf18ba, scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:01.646086 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a45de-41b7-4b05-9666-dbbae8f98bf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:01.647850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36a45de-41b7-4b05-9666-dbbae8f98bf1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:01.654662 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 187/367] 10.222.0.39 () {56 vars in 1376 bytes} [Thu Jul 28 22:15:01 2022] GET /identity/v3/role_assignments?group.id=62ee23836ebe49419e6178e20ed8a2ec&scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29&role.id=4146966c23674402aebb9a5b21cf18ba => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:01.655299 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a45de-41b7-4b05-9666-dbbae8f98bf1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:01.762904 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ad337f-d7fb-41f5-ab14-a05832ebe8ef None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:01.770311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:01.797577 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36a45de-41b7-4b05-9666-dbbae8f98bf1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:01.821270 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e8874-b075-41bf-8d8f-aec8d972bd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:01.821270 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e8874-b075-41bf-8d8f-aec8d972bd96 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:01.821569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e8874-b075-41bf-8d8f-aec8d972bd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:01.821822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18e8874-b075-41bf-8d8f-aec8d972bd96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:01.824314 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e8874-b075-41bf-8d8f-aec8d972bd96 None admin] RBAC: Adding query filter params (group.id=62ee23836ebe49419e6178e20ed8a2ec, role.id=4912a35790fe4d4297540156083d81d1, scope.project.id=e5b44df13345468b90906e747c09a977) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:01.824314 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e8874-b075-41bf-8d8f-aec8d972bd96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:01.824806 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18e8874-b075-41bf-8d8f-aec8d972bd96 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:01.830757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 181/368] 10.222.0.39 () {56 vars in 1376 bytes} [Thu Jul 28 22:15:01 2022] GET /identity/v3/role_assignments?group.id=62ee23836ebe49419e6178e20ed8a2ec&scope.project.id=e5b44df13345468b90906e747c09a977&role.id=4912a35790fe4d4297540156083d81d1 => generated 260 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:01.831370 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e8874-b075-41bf-8d8f-aec8d972bd96 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:01.844641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58f366-e634-4ed0-9ef2-ba142004e1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:01.845434 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58f366-e634-4ed0-9ef2-ba142004e1df None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:01.845852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58f366-e634-4ed0-9ef2-ba142004e1df None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:01.846140 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e58f366-e634-4ed0-9ef2-ba142004e1df None admin] PATH_INFO: `/v3/groups/62ee23836ebe49419e6178e20ed8a2ec` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:01.847551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e58f366-e634-4ed0-9ef2-ba142004e1df None admin] RBAC: Authorizing `identity:get_group(group_id=62ee23836ebe49419e6178e20ed8a2ec)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:01.849055 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e58f366-e634-4ed0-9ef2-ba142004e1df None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:01.850853 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 188/369] 10.222.0.39 () {56 vars in 1180 bytes} [Thu Jul 28 22:15:01 2022] GET /identity/v3/groups/62ee23836ebe49419e6178e20ed8a2ec => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:01.851637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58f366-e634-4ed0-9ef2-ba142004e1df None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:01.929883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:01.980543 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18e8874-b075-41bf-8d8f-aec8d972bd96 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:01.986536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e58f366-e634-4ed0-9ef2-ba142004e1df None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:02.031644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a3e142-c7d3-444f-9907-6319c090637c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:02.031850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97354b-84ba-40c5-b8a8-8531a1f04849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:02.032276 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3e142-c7d3-444f-9907-6319c090637c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:02.032484 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97354b-84ba-40c5-b8a8-8531a1f04849 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:02.032582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3e142-c7d3-444f-9907-6319c090637c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:02.032775 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97354b-84ba-40c5-b8a8-8531a1f04849 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:02.032869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a3e142-c7d3-444f-9907-6319c090637c None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:02.033062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c97354b-84ba-40c5-b8a8-8531a1f04849 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:02.033517 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a3e142-c7d3-444f-9907-6319c090637c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:02.033852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97354b-84ba-40c5-b8a8-8531a1f04849 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:02.033946 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a3e142-c7d3-444f-9907-6319c090637c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:02.034161 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97354b-84ba-40c5-b8a8-8531a1f04849 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:02.035214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a3e142-c7d3-444f-9907-6319c090637c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:02.035535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c97354b-84ba-40c5-b8a8-8531a1f04849 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:02.050864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 189/370] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:15:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:02.050864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97354b-84ba-40c5-b8a8-8531a1f04849 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:02.053937 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 182/371] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:02.055053 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a3e142-c7d3-444f-9907-6319c090637c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:02.077036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:02.181896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c97354b-84ba-40c5-b8a8-8531a1f04849 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:02.207029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a3e142-c7d3-444f-9907-6319c090637c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:02.217809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ccd57-2845-4688-b1f5-74699819eca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:02.218431 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ccd57-2845-4688-b1f5-74699819eca5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:02.218702 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ccd57-2845-4688-b1f5-74699819eca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:02.218977 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ccd57-2845-4688-b1f5-74699819eca5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:02.219769 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ccd57-2845-4688-b1f5-74699819eca5 None admin] RBAC: Adding query filter params (group.id=62ee23836ebe49419e6178e20ed8a2ec, role.id=4912a35790fe4d4297540156083d81d1, scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:02.220077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ccd57-2845-4688-b1f5-74699819eca5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:02.221502 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ccd57-2845-4688-b1f5-74699819eca5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:02.233829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 190/372] 10.222.0.39 () {56 vars in 1376 bytes} [Thu Jul 28 22:15:02 2022] GET /identity/v3/role_assignments?group.id=62ee23836ebe49419e6178e20ed8a2ec&scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29&role.id=4912a35790fe4d4297540156083d81d1 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:02.234821 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ccd57-2845-4688-b1f5-74699819eca5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:02.240291 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:02.247394 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657173b-88b2-404d-a802-568a578c3cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:02.248052 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657173b-88b2-404d-a802-568a578c3cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:02.248366 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657173b-88b2-404d-a802-568a578c3cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:02.249630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0657173b-88b2-404d-a802-568a578c3cd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:02.356711 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657173b-88b2-404d-a802-568a578c3cd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:02.358264 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0657173b-88b2-404d-a802-568a578c3cd8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:02.414083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:02.451354 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ccd57-2845-4688-b1f5-74699819eca5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:02.492418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5f08a-160e-428a-83f5-c72d2990a0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:02.493055 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5f08a-160e-428a-83f5-c72d2990a0af None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:02.493343 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5f08a-160e-428a-83f5-c72d2990a0af None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:02.493647 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5f08a-160e-428a-83f5-c72d2990a0af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:02.529683 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 183/373] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:15:02.536580 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657173b-88b2-404d-a802-568a578c3cd8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:02.570235 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5f08a-160e-428a-83f5-c72d2990a0af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:02.571768 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5f08a-160e-428a-83f5-c72d2990a0af None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:02.591719 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:02.703294 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:02.726782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 191/374] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:15:02.732800 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5f08a-160e-428a-83f5-c72d2990a0af None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:02.751111 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0657173b-88b2-404d-a802-568a578c3cd8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:02.818853 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a376a8c9-f07a-4aec-847f-01a5f09c7e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:02.819610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376a8c9-f07a-4aec-847f-01a5f09c7e25 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:02.819790 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376a8c9-f07a-4aec-847f-01a5f09c7e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:02.820039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376a8c9-f07a-4aec-847f-01a5f09c7e25 None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:02.827325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a376a8c9-f07a-4aec-847f-01a5f09c7e25 None admin] RBAC: Authorizing `identity:get_project(project_id=e5b44df13345468b90906e747c09a977)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:02.828832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a376a8c9-f07a-4aec-847f-01a5f09c7e25 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:02.832885 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 184/375] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:15:02 2022] GET /identity/v3/projects/e5b44df13345468b90906e747c09a977 => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:02.840739 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a376a8c9-f07a-4aec-847f-01a5f09c7e25 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:02.930453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5f08a-160e-428a-83f5-c72d2990a0af None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:02.991436 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63f288-5722-45b2-8c96-98352b5fd9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:02.991995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63f288-5722-45b2-8c96-98352b5fd9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:02.992276 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63f288-5722-45b2-8c96-98352b5fd9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:02.992562 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63f288-5722-45b2-8c96-98352b5fd9f6 None admin] PATH_INFO: `/v3/projects/26d9130b7556481bb766f1b06a63e1e5` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:02.993766 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63f288-5722-45b2-8c96-98352b5fd9f6 None admin] RBAC: Authorizing `identity:get_project(project_id=26d9130b7556481bb766f1b06a63e1e5)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:02.995562 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63f288-5722-45b2-8c96-98352b5fd9f6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:03.001576 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 192/376] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:15:02 2022] GET /identity/v3/projects/26d9130b7556481bb766f1b06a63e1e5 => generated 344 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:03.005022 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63f288-5722-45b2-8c96-98352b5fd9f6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:03.006116 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a376a8c9-f07a-4aec-847f-01a5f09c7e25 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:03.007835 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:03.081282 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9cf97c-0ea5-4cc4-83e3-a9adf13f6360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:03.081976 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cf97c-0ea5-4cc4-83e3-a9adf13f6360 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:03.082254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cf97c-0ea5-4cc4-83e3-a9adf13f6360 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:03.082524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9cf97c-0ea5-4cc4-83e3-a9adf13f6360 None admin] PATH_INFO: `/v3/groups/62ee23836ebe49419e6178e20ed8a2ec` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:03.083762 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9cf97c-0ea5-4cc4-83e3-a9adf13f6360 None admin] RBAC: Authorizing `identity:get_group(group_id=62ee23836ebe49419e6178e20ed8a2ec)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:03.085237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9cf97c-0ea5-4cc4-83e3-a9adf13f6360 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:03.087344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 185/377] 10.222.0.39 () {56 vars in 1180 bytes} [Thu Jul 28 22:15:02 2022] GET /identity/v3/groups/62ee23836ebe49419e6178e20ed8a2ec => generated 226 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:03.094261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9cf97c-0ea5-4cc4-83e3-a9adf13f6360 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:03.111967 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:03.236631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63f288-5722-45b2-8c96-98352b5fd9f6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:03.283078 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9cf97c-0ea5-4cc4-83e3-a9adf13f6360 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:03.321240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92676372-af0e-4534-8159-a34d72c63fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:03.321945 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92676372-af0e-4534-8159-a34d72c63fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:03.322246 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92676372-af0e-4534-8159-a34d72c63fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:03.322492 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92676372-af0e-4534-8159-a34d72c63fe5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:03.324061 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92676372-af0e-4534-8159-a34d72c63fe5 None admin] RBAC: Adding query filter params (group.id=62ee23836ebe49419e6178e20ed8a2ec, role.id=4146966c23674402aebb9a5b21cf18ba, scope.project.id=e5b44df13345468b90906e747c09a977) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:03.324363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92676372-af0e-4534-8159-a34d72c63fe5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:03.325185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe34fe-61de-4fe4-a4ed-2c4a122ab4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:03.325823 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92676372-af0e-4534-8159-a34d72c63fe5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:03.326341 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe34fe-61de-4fe4-a4ed-2c4a122ab4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:03.326819 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe34fe-61de-4fe4-a4ed-2c4a122ab4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:03.327290 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe34fe-61de-4fe4-a4ed-2c4a122ab4e2 None admin] PATH_INFO: `/v3/groups/f8b9f199082546748b71117a9a5c9bb4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:03.333976 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 186/378] 10.222.0.39 () {56 vars in 1376 bytes} [Thu Jul 28 22:15:03 2022] GET /identity/v3/role_assignments?group.id=62ee23836ebe49419e6178e20ed8a2ec&scope.project.id=e5b44df13345468b90906e747c09a977&role.id=4146966c23674402aebb9a5b21cf18ba => generated 260 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:03.340667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe34fe-61de-4fe4-a4ed-2c4a122ab4e2 None admin] RBAC: Authorizing `identity:get_group(group_id=f8b9f199082546748b71117a9a5c9bb4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:03.342240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe34fe-61de-4fe4-a4ed-2c4a122ab4e2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:03.344020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 193/379] 10.222.0.39 () {56 vars in 1180 bytes} [Thu Jul 28 22:15:03 2022] GET /identity/v3/groups/f8b9f199082546748b71117a9a5c9bb4 => generated 229 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:03.350512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92676372-af0e-4534-8159-a34d72c63fe5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:03.393326 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:03.491870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92676372-af0e-4534-8159-a34d72c63fe5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:03.535696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdc01f6-74f3-4fc5-9ebc-177217db9ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:03.536243 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc01f6-74f3-4fc5-9ebc-177217db9ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:03.536521 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc01f6-74f3-4fc5-9ebc-177217db9ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:03.536835 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdc01f6-74f3-4fc5-9ebc-177217db9ac6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:03.537666 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc01f6-74f3-4fc5-9ebc-177217db9ac6 None admin] RBAC: Adding query filter params (group.id=f8b9f199082546748b71117a9a5c9bb4, role.id=aded7bd43e974686b3c23bd9d4255b24, scope.project.id=26d9130b7556481bb766f1b06a63e1e5) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:03.537974 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc01f6-74f3-4fc5-9ebc-177217db9ac6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:03.539413 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdc01f6-74f3-4fc5-9ebc-177217db9ac6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:03.546330 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 187/380] 10.222.0.39 () {56 vars in 1376 bytes} [Thu Jul 28 22:15:03 2022] GET /identity/v3/role_assignments?group.id=f8b9f199082546748b71117a9a5c9bb4&scope.project.id=26d9130b7556481bb766f1b06a63e1e5&role.id=aded7bd43e974686b3c23bd9d4255b24 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:03.915888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4ecdc-dee9-47df-9cf8-49a03dbb6dac None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:03.918404 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4ecdc-dee9-47df-9cf8-49a03dbb6dac None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:03.918404 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4ecdc-dee9-47df-9cf8-49a03dbb6dac None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:03.918896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 194/381] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:03.924335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108e189-c61f-4d87-998f-09e4d0765044 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:03.925114 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108e189-c61f-4d87-998f-09e4d0765044 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:03.925558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108e189-c61f-4d87-998f-09e4d0765044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:03.950538 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2108e189-c61f-4d87-998f-09e4d0765044 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:03.952004 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2108e189-c61f-4d87-998f-09e4d0765044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:04.044002 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 188/382] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:15:04.069132 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4ecdc-dee9-47df-9cf8-49a03dbb6dac None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:04.202879 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e4ecdc-dee9-47df-9cf8-49a03dbb6dac None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:04.220525 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:04.249331 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293fedc-7fa7-4d4d-b1ad-15b2a37017da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:04.250005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293fedc-7fa7-4d4d-b1ad-15b2a37017da None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:04.250291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293fedc-7fa7-4d4d-b1ad-15b2a37017da None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:04.250570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293fedc-7fa7-4d4d-b1ad-15b2a37017da None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:04.269566 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293fedc-7fa7-4d4d-b1ad-15b2a37017da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:04.274687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-a293fedc-7fa7-4d4d-b1ad-15b2a37017da None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:15:04.274687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:04.274687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:04.274687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:04.274687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:04.274687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:04.274687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:04.274687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:04.274687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:04.274687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:04.274687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:04.274687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:04.274687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:04.274687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:04.274687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:04.274687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:04.275617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:04.275617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:04.275617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:04.275617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:04.275617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:04.275617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:04.275617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:04.275617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:04.275617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:04.275617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:04.275617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:04.275617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:04.275617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:04.275617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:04.275617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:04.275617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:04.277088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:04.277088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:04.277088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:04.277088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:04.277088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:04.277088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:04.277088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:04.277088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:04.277088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:04.277088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:04.277088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:15:04.277088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:04.277922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 195/383] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:15:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:04.282814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108e189-c61f-4d87-998f-09e4d0765044 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:04.339190 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:04.403246 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98f987-e303-493a-ae75-5fcf392fe937 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:04.403572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98f987-e303-493a-ae75-5fcf392fe937 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:04.403820 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e98f987-e303-493a-ae75-5fcf392fe937 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:04.405484 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 196/384] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:15:04.412053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0c9a4-9186-4a79-b821-a220f5dae59b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:04.412343 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0c9a4-9186-4a79-b821-a220f5dae59b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:04.412628 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef0c9a4-9186-4a79-b821-a220f5dae59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:04.446760 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108e189-c61f-4d87-998f-09e4d0765044 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:04.454649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8ef0c9a4-9186-4a79-b821-a220f5dae59b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:04.456202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8ef0c9a4-9186-4a79-b821-a220f5dae59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:04.486855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257402c-5df8-4c42-8b62-56a149768177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:04.487489 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257402c-5df8-4c42-8b62-56a149768177 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:04.487766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257402c-5df8-4c42-8b62-56a149768177 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:04.488036 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257402c-5df8-4c42-8b62-56a149768177 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:04.488762 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257402c-5df8-4c42-8b62-56a149768177 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:04.489065 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257402c-5df8-4c42-8b62-56a149768177 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:04.490463 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5257402c-5df8-4c42-8b62-56a149768177 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:04.513884 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 189/385] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:04.522170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257402c-5df8-4c42-8b62-56a149768177 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:04.608000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 197/386] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:15:04.617103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cda7cc-c2fd-4df2-a198-974f241d69a1 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:04.617103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cda7cc-c2fd-4df2-a198-974f241d69a1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:04.617103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cda7cc-c2fd-4df2-a198-974f241d69a1 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:04.617103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 198/387] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:04.617103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f722086-7756-4114-9efb-eea8aca9a782 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:04.617103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f722086-7756-4114-9efb-eea8aca9a782 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:04.617564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f722086-7756-4114-9efb-eea8aca9a782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:04.634848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:04.641569 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5f722086-7756-4114-9efb-eea8aca9a782 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:04.642693 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5f722086-7756-4114-9efb-eea8aca9a782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:04.658181 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257402c-5df8-4c42-8b62-56a149768177 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:04.704124 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c54cd-dc4d-4358-a1a3-a1ef9616f584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:04.705910 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c54cd-dc4d-4358-a1a3-a1ef9616f584 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:04.706441 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c54cd-dc4d-4358-a1a3-a1ef9616f584 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:04.707151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546c54cd-dc4d-4358-a1a3-a1ef9616f584 None admin] PATH_INFO: `/v3/groups/62ee23836ebe49419e6178e20ed8a2ec` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:04.708814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c54cd-dc4d-4358-a1a3-a1ef9616f584 None admin] RBAC: Authorizing `identity:get_group(group_id=62ee23836ebe49419e6178e20ed8a2ec)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:04.711283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546c54cd-dc4d-4358-a1a3-a1ef9616f584 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:04.714852 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 190/388] 10.222.0.39 () {56 vars in 1180 bytes} [Thu Jul 28 22:15:04 2022] GET /identity/v3/groups/62ee23836ebe49419e6178e20ed8a2ec => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:04.716079 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c54cd-dc4d-4358-a1a3-a1ef9616f584 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:04.734141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 199/389] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:15:04.734998 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f722086-7756-4114-9efb-eea8aca9a782 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:04.779875 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:04.870276 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f722086-7756-4114-9efb-eea8aca9a782 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:04.879043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546c54cd-dc4d-4358-a1a3-a1ef9616f584 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:04.907298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed452080-a509-4d82-a50d-589bd298a53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:04.907940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed452080-a509-4d82-a50d-589bd298a53e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:04.908223 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed452080-a509-4d82-a50d-589bd298a53e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:04.908494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed452080-a509-4d82-a50d-589bd298a53e None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:04.909707 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed452080-a509-4d82-a50d-589bd298a53e None admin] RBAC: Authorizing `identity:get_project(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:04.911215 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed452080-a509-4d82-a50d-589bd298a53e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:04.913067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 200/390] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:15:04 2022] GET /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:04.918551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed452080-a509-4d82-a50d-589bd298a53e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:04.923184 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb41683-74bc-4484-8175-da0a406326ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:04.924088 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb41683-74bc-4484-8175-da0a406326ca None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:04.924561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb41683-74bc-4484-8175-da0a406326ca None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:04.925461 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb41683-74bc-4484-8175-da0a406326ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:04.937642 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb41683-74bc-4484-8175-da0a406326ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:04.943493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-2cb41683-74bc-4484-8175-da0a406326ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:15:04.943493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:04.943493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:04.943493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:04.943493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:04.943493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:04.943493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:04.943493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:04.943493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:04.943493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:04.943493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:04.943493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:04.943493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:04.943493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:04.943493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:04.943493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:04.944363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:04.944363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:04.944363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:04.944363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:04.944363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:04.944363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:04.944363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:04.944363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:04.944363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:04.944363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:04.944363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:04.944363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:04.944363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:04.944363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:04.944363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:04.944363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:04.945251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:04.945251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:04.945251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:04.945251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:04.945251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:04.945251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:04.945251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:04.945251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:04.945251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:04.945251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:04.945251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:15:04.945251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:04.953302 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 191/391] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:15:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:04.954760 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb41683-74bc-4484-8175-da0a406326ca None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:04.986042 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:05.029828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:05.032761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed452080-a509-4d82-a50d-589bd298a53e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:05.069784 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fb516-99a2-4c64-ab3b-ab6d4c0ca7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:05.070568 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fb516-99a2-4c64-ab3b-ab6d4c0ca7ac None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:05.070953 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fb516-99a2-4c64-ab3b-ab6d4c0ca7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:05.071442 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fb516-99a2-4c64-ab3b-ab6d4c0ca7ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:05.079778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb41683-74bc-4484-8175-da0a406326ca None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:05.083401 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873fb516-99a2-4c64-ab3b-ab6d4c0ca7ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:05.088307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-873fb516-99a2-4c64-ab3b-ab6d4c0ca7ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:15:05.088307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:05.088307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:05.088307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:05.088307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:05.088307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:05.088307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:05.088307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:05.088307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:05.088307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:05.088307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:05.088307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:05.088307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:05.088307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:05.088307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:05.088307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:05.089407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:05.089407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:05.089407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:05.089407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:05.089407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:05.089407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:05.089407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:05.089407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:05.089407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:05.089407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:05.089407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:05.089407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:05.089407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:05.089407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:05.089407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:05.089407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:05.090432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:05.090432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:05.090432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:05.090432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:05.090432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:05.090432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:05.090432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:05.090432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:05.090432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:05.090432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:05.090432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:15:05.090432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:05.096425 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 201/392] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:15:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:05.096425 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fb516-99a2-4c64-ab3b-ab6d4c0ca7ac None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:05.131670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0a895-c03d-4493-99f0-feac487f7d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:05.133466 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0a895-c03d-4493-99f0-feac487f7d77 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:05.133954 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0a895-c03d-4493-99f0-feac487f7d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:05.134425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c0a895-c03d-4493-99f0-feac487f7d77 None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29/groups/62ee23836ebe49419e6178e20ed8a2ec/roles/4146966c23674402aebb9a5b21cf18ba` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:05.137881 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0a895-c03d-4493-99f0-feac487f7d77 None admin] RBAC: Authorizing `identity:create_grant(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29, group_id=62ee23836ebe49419e6178e20ed8a2ec, role_id=4146966c23674402aebb9a5b21cf18ba)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:05.143511 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c0a895-c03d-4493-99f0-feac487f7d77 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:05.167081 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 192/393] 10.222.0.39 () {58 vars in 1442 bytes} [Thu Jul 28 22:15:04 2022] PUT /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29/groups/62ee23836ebe49419e6178e20ed8a2ec/roles/4146966c23674402aebb9a5b21cf18ba => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:15:05.168336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0a895-c03d-4493-99f0-feac487f7d77 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:05.264471 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:05.296474 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fb516-99a2-4c64-ab3b-ab6d4c0ca7ac None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:05.301634 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c0a895-c03d-4493-99f0-feac487f7d77 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:05.347477 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7ea68-a382-4bbc-8ef7-eab9717075b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:05.348509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7ea68-a382-4bbc-8ef7-eab9717075b1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:05.350638 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7ea68-a382-4bbc-8ef7-eab9717075b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:05.351129 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7ea68-a382-4bbc-8ef7-eab9717075b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:05.352108 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7ea68-a382-4bbc-8ef7-eab9717075b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:05.353628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7ea68-a382-4bbc-8ef7-eab9717075b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:05.356110 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7ea68-a382-4bbc-8ef7-eab9717075b1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:05.378391 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 193/394] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:15:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:05.380761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243b599-f90d-4d12-a99a-cedef597aecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:05.381654 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243b599-f90d-4d12-a99a-cedef597aecb None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:05.382131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243b599-f90d-4d12-a99a-cedef597aecb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:05.382620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243b599-f90d-4d12-a99a-cedef597aecb None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:05.383538 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243b599-f90d-4d12-a99a-cedef597aecb None admin] RBAC: Adding query filter params (name=member) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:05.384635 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7ea68-a382-4bbc-8ef7-eab9717075b1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:05.387970 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243b599-f90d-4d12-a99a-cedef597aecb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:05.389934 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243b599-f90d-4d12-a99a-cedef597aecb None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:05.411720 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 202/395] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:15:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:05.418329 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243b599-f90d-4d12-a99a-cedef597aecb None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:05.454090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:05.531563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7ea68-a382-4bbc-8ef7-eab9717075b1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:05.558450 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243b599-f90d-4d12-a99a-cedef597aecb None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:05.567952 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b41d9-0f8f-4768-a75f-f7ba3f748ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:05.568602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b41d9-0f8f-4768-a75f-f7ba3f748ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:05.569173 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b41d9-0f8f-4768-a75f-f7ba3f748ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:05.569173 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b41d9-0f8f-4768-a75f-f7ba3f748ff5 None admin] PATH_INFO: `/v3/groups/62ee23836ebe49419e6178e20ed8a2ec` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:05.570312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b41d9-0f8f-4768-a75f-f7ba3f748ff5 None admin] RBAC: Authorizing `identity:get_group(group_id=62ee23836ebe49419e6178e20ed8a2ec)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:05.571881 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b41d9-0f8f-4768-a75f-f7ba3f748ff5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:05.574460 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 194/396] 10.222.0.39 () {56 vars in 1180 bytes} [Thu Jul 28 22:15:05 2022] GET /identity/v3/groups/62ee23836ebe49419e6178e20ed8a2ec => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:05.579883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b41d9-0f8f-4768-a75f-f7ba3f748ff5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:05.583668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:05.607295 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400516c-2edb-404c-834a-51d50d9f966a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:05.608071 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400516c-2edb-404c-834a-51d50d9f966a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:05.608424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400516c-2edb-404c-834a-51d50d9f966a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:05.608769 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7400516c-2edb-404c-834a-51d50d9f966a None admin] PATH_INFO: `/v3/groups/62ee23836ebe49419e6178e20ed8a2ec` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:05.610029 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7400516c-2edb-404c-834a-51d50d9f966a None admin] RBAC: Authorizing `identity:get_group(group_id=62ee23836ebe49419e6178e20ed8a2ec)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:05.611687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7400516c-2edb-404c-834a-51d50d9f966a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:05.613517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 203/397] 10.222.0.39 () {56 vars in 1180 bytes} [Thu Jul 28 22:15:05 2022] GET /identity/v3/groups/62ee23836ebe49419e6178e20ed8a2ec => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:05.622556 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400516c-2edb-404c-834a-51d50d9f966a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:05.697095 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b41d9-0f8f-4768-a75f-f7ba3f748ff5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:05.744111 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:05.755553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7400516c-2edb-404c-834a-51d50d9f966a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:05.786071 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae35086-2a70-4369-89a5-e77ff9f51b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:05.786820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae35086-2a70-4369-89a5-e77ff9f51b1d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:05.787109 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae35086-2a70-4369-89a5-e77ff9f51b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:05.787439 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae35086-2a70-4369-89a5-e77ff9f51b1d None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:05.788668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae35086-2a70-4369-89a5-e77ff9f51b1d None admin] RBAC: Authorizing `identity:get_project(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:05.790372 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae35086-2a70-4369-89a5-e77ff9f51b1d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:05.793072 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 195/398] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:15:05 2022] GET /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:05.794375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def028fb-6c84-48e0-9d59-1a83b3b32b93 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:05.794657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def028fb-6c84-48e0-9d59-1a83b3b32b93 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:05.794930 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def028fb-6c84-48e0-9d59-1a83b3b32b93 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:05.795030 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db32a434-404a-4a6f-85ec-98ecfc9f176e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:05.795638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32a434-404a-4a6f-85ec-98ecfc9f176e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:05.795923 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32a434-404a-4a6f-85ec-98ecfc9f176e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:05.796217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 196/399] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:05.796422 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32a434-404a-4a6f-85ec-98ecfc9f176e None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:05.797662 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db32a434-404a-4a6f-85ec-98ecfc9f176e None admin] RBAC: Authorizing `identity:get_project(project_id=e5b44df13345468b90906e747c09a977)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:05.799095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db32a434-404a-4a6f-85ec-98ecfc9f176e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:05.800979 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 204/400] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:15:05 2022] GET /identity/v3/projects/e5b44df13345468b90906e747c09a977 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:05.801086 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def028fb-6c84-48e0-9d59-1a83b3b32b93 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:05.805948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b842-b349-461f-a646-8ac41a582e83 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:05.806235 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b842-b349-461f-a646-8ac41a582e83 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:05.806514 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d798b842-b349-461f-a646-8ac41a582e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:05.829626 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d798b842-b349-461f-a646-8ac41a582e83 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:05.831114 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d798b842-b349-461f-a646-8ac41a582e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:05.901974 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:05.925620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 205/401] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:05.926493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798b842-b349-461f-a646-8ac41a582e83 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:05.932924 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def028fb-6c84-48e0-9d59-1a83b3b32b93 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:05.977790 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd058a8-a823-4d33-8306-62df85309f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:05.978621 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd058a8-a823-4d33-8306-62df85309f91 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:05.979691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd058a8-a823-4d33-8306-62df85309f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:05.980121 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd058a8-a823-4d33-8306-62df85309f91 None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29/groups/62ee23836ebe49419e6178e20ed8a2ec/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:05.982434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd058a8-a823-4d33-8306-62df85309f91 None admin] RBAC: Authorizing `identity:create_grant(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29, group_id=62ee23836ebe49419e6178e20ed8a2ec, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:05.984043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd058a8-a823-4d33-8306-62df85309f91 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:06.000083 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 197/402] 10.222.0.39 () {58 vars in 1442 bytes} [Thu Jul 28 22:15:05 2022] PUT /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29/groups/62ee23836ebe49419e6178e20ed8a2ec/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:15:06.001061 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd058a8-a823-4d33-8306-62df85309f91 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:06.058852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d798b842-b349-461f-a646-8ac41a582e83 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:06.061926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:06.100078 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8032ac-b16d-4712-af6b-84305aee404c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:06.100923 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8032ac-b16d-4712-af6b-84305aee404c None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:06.101282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8032ac-b16d-4712-af6b-84305aee404c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:06.101798 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8032ac-b16d-4712-af6b-84305aee404c None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977/groups/62ee23836ebe49419e6178e20ed8a2ec/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:06.105151 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8032ac-b16d-4712-af6b-84305aee404c None admin] RBAC: Authorizing `identity:create_grant(project_id=e5b44df13345468b90906e747c09a977, group_id=62ee23836ebe49419e6178e20ed8a2ec, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:06.106378 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8032ac-b16d-4712-af6b-84305aee404c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:06.125573 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 206/403] 10.222.0.39 () {58 vars in 1442 bytes} [Thu Jul 28 22:15:05 2022] PUT /identity/v3/projects/e5b44df13345468b90906e747c09a977/groups/62ee23836ebe49419e6178e20ed8a2ec/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:15:06.127824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0662836-4817-47d0-b166-e7127985f031 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:06.128378 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0662836-4817-47d0-b166-e7127985f031 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:06.128866 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0662836-4817-47d0-b166-e7127985f031 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:06.134960 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 207/404] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:06.137557 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cd074-b467-411d-8cba-b6a715bd2928 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:06.138147 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cd074-b467-411d-8cba-b6a715bd2928 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:06.138619 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89cd074-b467-411d-8cba-b6a715bd2928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:06.142633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd058a8-a823-4d33-8306-62df85309f91 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:06.161836 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a89cd074-b467-411d-8cba-b6a715bd2928 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:06.163442 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a89cd074-b467-411d-8cba-b6a715bd2928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:06.184517 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:06.193458 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb115812-635c-441b-9a65-2a38c7594344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:06.194163 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb115812-635c-441b-9a65-2a38c7594344 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:06.194861 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb115812-635c-441b-9a65-2a38c7594344 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:06.195162 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb115812-635c-441b-9a65-2a38c7594344 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:06.214651 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb115812-635c-441b-9a65-2a38c7594344 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:06.221820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-cb115812-635c-441b-9a65-2a38c7594344 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:15:06.221820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:06.221820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:06.221820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:06.221820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:06.221820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:06.221820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:06.221820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:06.221820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:06.221820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:06.221820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:06.221820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:06.221820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:06.221820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:06.221820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:06.221820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:06.222804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:06.222804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:06.222804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:06.222804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:06.222804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:06.222804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:06.222804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:06.222804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:06.222804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:06.222804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:06.222804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:06.222804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:06.222804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:06.222804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:06.222804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:06.222804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:06.223636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:06.223636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:06.223636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:06.223636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:06.223636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:06.223636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:06.223636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:06.223636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:06.223636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:06.223636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:06.223636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:15:06.223636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:06.225231 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 198/405] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:15:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:06.231541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb115812-635c-441b-9a65-2a38c7594344 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:06.262651 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 208/406] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:06.282597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89cd074-b467-411d-8cba-b6a715bd2928 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:06.361017 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb115812-635c-441b-9a65-2a38c7594344 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:06.393951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:06.417904 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf9e60-622e-4066-b92f-440fd8677b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:06.419346 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89cd074-b467-411d-8cba-b6a715bd2928 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:06.419669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf9e60-622e-4066-b92f-440fd8677b49 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:06.420450 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf9e60-622e-4066-b92f-440fd8677b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:06.421196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf9e60-622e-4066-b92f-440fd8677b49 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:06.422737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf9e60-622e-4066-b92f-440fd8677b49 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:06.423584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf9e60-622e-4066-b92f-440fd8677b49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:06.426406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf9e60-622e-4066-b92f-440fd8677b49 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:06.442156 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 199/407] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:06.449737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf9e60-622e-4066-b92f-440fd8677b49 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:06.469994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1d62c-2e67-496b-9ad9-45ca11ae7e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:06.472524 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1d62c-2e67-496b-9ad9-45ca11ae7e93 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:06.473267 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1d62c-2e67-496b-9ad9-45ca11ae7e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:06.475158 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1d62c-2e67-496b-9ad9-45ca11ae7e93 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:06.501790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc1d62c-2e67-496b-9ad9-45ca11ae7e93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:06.512159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-8fc1d62c-2e67-496b-9ad9-45ca11ae7e93 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:15:06.512159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:06.512159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:06.512159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:06.512159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:06.512159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:06.512159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:06.512159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:06.512159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:06.512159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:06.512159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:06.512159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:06.512159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:06.512159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:06.512159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:06.512159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:06.513087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:06.513087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:06.513087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:06.513087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:06.513087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:06.513087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:06.513087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:06.513087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:06.513087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:06.513087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:06.513087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:06.513087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:06.513087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:06.513087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:06.513087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:06.513087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:06.513954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:06.513954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:06.513954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:06.513954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:06.513954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:06.513954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:06.513954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:06.513954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:06.513954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:06.513954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:06.513954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:15:06.513954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:06.520485 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 209/408] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:15:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:15:06.531627 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1d62c-2e67-496b-9ad9-45ca11ae7e93 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:06.561293 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:06.612714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf9e60-622e-4066-b92f-440fd8677b49 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:06.678943 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b3916-5008-46f9-be90-5ec8b984f376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:06.679603 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b3916-5008-46f9-be90-5ec8b984f376 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:06.679888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b3916-5008-46f9-be90-5ec8b984f376 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:06.680163 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340b3916-5008-46f9-be90-5ec8b984f376 None admin] PATH_INFO: `/v3/groups/62ee23836ebe49419e6178e20ed8a2ec` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:06.682859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b3916-5008-46f9-be90-5ec8b984f376 None admin] RBAC: Authorizing `identity:get_group(group_id=62ee23836ebe49419e6178e20ed8a2ec)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:06.684486 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340b3916-5008-46f9-be90-5ec8b984f376 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:06.686151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 200/409] 10.222.0.39 () {56 vars in 1180 bytes} [Thu Jul 28 22:15:06 2022] GET /identity/v3/groups/62ee23836ebe49419e6178e20ed8a2ec => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:06.692052 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b3916-5008-46f9-be90-5ec8b984f376 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:06.720366 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc1d62c-2e67-496b-9ad9-45ca11ae7e93 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:06.761625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bf9c0-39ed-42e2-89f4-84e93dcb52d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:06.762475 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bf9c0-39ed-42e2-89f4-84e93dcb52d0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:06.762853 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bf9c0-39ed-42e2-89f4-84e93dcb52d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:06.763270 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bf9c0-39ed-42e2-89f4-84e93dcb52d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:06.769555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bf9c0-39ed-42e2-89f4-84e93dcb52d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:06.769555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bf9c0-39ed-42e2-89f4-84e93dcb52d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:06.769555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bf9c0-39ed-42e2-89f4-84e93dcb52d0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:06.772767 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:06.779119 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 210/410] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:15:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:06.785676 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bf9c0-39ed-42e2-89f4-84e93dcb52d0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:06.905624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bf9c0-39ed-42e2-89f4-84e93dcb52d0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:06.916276 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340b3916-5008-46f9-be90-5ec8b984f376 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:06.951291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375f95b-08d4-4028-9a4d-2ece3af5bff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:06.951943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375f95b-08d4-4028-9a4d-2ece3af5bff6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:06.952224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375f95b-08d4-4028-9a4d-2ece3af5bff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:06.952500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375f95b-08d4-4028-9a4d-2ece3af5bff6 None admin] PATH_INFO: `/v3/groups/f8b9f199082546748b71117a9a5c9bb4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:06.953879 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375f95b-08d4-4028-9a4d-2ece3af5bff6 None admin] RBAC: Authorizing `identity:get_group(group_id=f8b9f199082546748b71117a9a5c9bb4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:06.961399 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:06.961880 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f375f95b-08d4-4028-9a4d-2ece3af5bff6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:06.964453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 211/411] 10.222.0.39 () {56 vars in 1180 bytes} [Thu Jul 28 22:15:06 2022] GET /identity/v3/groups/f8b9f199082546748b71117a9a5c9bb4 => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:06.971807 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375f95b-08d4-4028-9a4d-2ece3af5bff6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:06.988126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c00e1c-9242-4663-b1be-836f912224b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:06.988842 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c00e1c-9242-4663-b1be-836f912224b8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:06.989124 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c00e1c-9242-4663-b1be-836f912224b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:06.989493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c00e1c-9242-4663-b1be-836f912224b8 None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:06.990732 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c00e1c-9242-4663-b1be-836f912224b8 None admin] RBAC: Authorizing `identity:get_project(project_id=e5b44df13345468b90906e747c09a977)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:06.992400 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c00e1c-9242-4663-b1be-836f912224b8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:06.994105 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 201/412] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:15:06 2022] GET /identity/v3/projects/e5b44df13345468b90906e747c09a977 => generated 302 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:07.002067 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c00e1c-9242-4663-b1be-836f912224b8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:07.109832 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:07.174613 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375f95b-08d4-4028-9a4d-2ece3af5bff6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:07.195422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c00e1c-9242-4663-b1be-836f912224b8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:07.233448 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ace203-c0c5-4746-93e7-d690f059b476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:07.234334 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ace203-c0c5-4746-93e7-d690f059b476 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:07.234723 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ace203-c0c5-4746-93e7-d690f059b476 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:07.235092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ace203-c0c5-4746-93e7-d690f059b476 None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977/groups/62ee23836ebe49419e6178e20ed8a2ec/roles/4146966c23674402aebb9a5b21cf18ba` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:07.237826 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ace203-c0c5-4746-93e7-d690f059b476 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5b44df13345468b90906e747c09a977, group_id=62ee23836ebe49419e6178e20ed8a2ec, role_id=4146966c23674402aebb9a5b21cf18ba)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:07.239363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ace203-c0c5-4746-93e7-d690f059b476 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:07.246863 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46446e91-14f0-4332-b351-d7441d93d24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:07.247760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46446e91-14f0-4332-b351-d7441d93d24d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:07.248131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46446e91-14f0-4332-b351-d7441d93d24d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:07.248504 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46446e91-14f0-4332-b351-d7441d93d24d None admin] PATH_INFO: `/v3/projects/26d9130b7556481bb766f1b06a63e1e5` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:07.250005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46446e91-14f0-4332-b351-d7441d93d24d None admin] RBAC: Authorizing `identity:get_project(project_id=26d9130b7556481bb766f1b06a63e1e5)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:07.251590 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46446e91-14f0-4332-b351-d7441d93d24d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:07.258792 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 202/413] 10.222.0.39 () {58 vars in 1442 bytes} [Thu Jul 28 22:15:07 2022] PUT /identity/v3/projects/e5b44df13345468b90906e747c09a977/groups/62ee23836ebe49419e6178e20ed8a2ec/roles/4146966c23674402aebb9a5b21cf18ba => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:15:07.261277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 212/414] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:15:06 2022] GET /identity/v3/projects/26d9130b7556481bb766f1b06a63e1e5 => generated 344 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:07.267600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ace203-c0c5-4746-93e7-d690f059b476 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:07.327294 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:07.379863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ace203-c0c5-4746-93e7-d690f059b476 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:07.417445 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8401b5fc-9773-49cf-92a1-6cc50e5f197e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:07.418090 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401b5fc-9773-49cf-92a1-6cc50e5f197e None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:07.418375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401b5fc-9773-49cf-92a1-6cc50e5f197e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:07.418654 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8401b5fc-9773-49cf-92a1-6cc50e5f197e None admin] PATH_INFO: `/v3/projects/26d9130b7556481bb766f1b06a63e1e5/groups/f8b9f199082546748b71117a9a5c9bb4/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:07.420691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8401b5fc-9773-49cf-92a1-6cc50e5f197e None admin] RBAC: Authorizing `identity:create_grant(project_id=26d9130b7556481bb766f1b06a63e1e5, group_id=f8b9f199082546748b71117a9a5c9bb4, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:07.422225 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8401b5fc-9773-49cf-92a1-6cc50e5f197e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:07.450135 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 203/415] 10.222.0.39 () {58 vars in 1442 bytes} [Thu Jul 28 22:15:07 2022] PUT /identity/v3/projects/26d9130b7556481bb766f1b06a63e1e5/groups/f8b9f199082546748b71117a9a5c9bb4/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:15:07.663563 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092983f9-e890-42b6-a803-402c69dad43d None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:07.663960 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092983f9-e890-42b6-a803-402c69dad43d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:07.664136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092983f9-e890-42b6-a803-402c69dad43d None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:07.665637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 213/416] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:07.671885 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed2c94-4e89-432c-8bf2-104dd0f9df39 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:07.672114 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed2c94-4e89-432c-8bf2-104dd0f9df39 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:07.672390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed2c94-4e89-432c-8bf2-104dd0f9df39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:07.692496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-13ed2c94-4e89-432c-8bf2-104dd0f9df39 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:07.693729 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-13ed2c94-4e89-432c-8bf2-104dd0f9df39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:07.766733 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 204/417] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:07.792227 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092983f9-e890-42b6-a803-402c69dad43d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:07.928469 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092983f9-e890-42b6-a803-402c69dad43d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:07.977711 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f09c83-b4ba-44fb-84e5-0a9863362be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:07.978356 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f09c83-b4ba-44fb-84e5-0a9863362be0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:07.978633 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f09c83-b4ba-44fb-84e5-0a9863362be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:07.979069 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f09c83-b4ba-44fb-84e5-0a9863362be0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:07.983886 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:07.995858 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f09c83-b4ba-44fb-84e5-0a9863362be0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:08.001139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-91f09c83-b4ba-44fb-84e5-0a9863362be0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:15:08.001139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:08.001139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:08.001139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:08.001139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:08.001139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:08.001139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:08.001139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:08.001139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:08.001139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:08.001139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:08.001139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:08.001139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:08.001139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:08.001139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:08.001139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:08.002062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:08.002062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:08.002062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:08.002062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:08.002062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:08.002062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:08.002062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:08.002062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:08.002062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:08.002062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:08.002062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:08.002062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:08.002062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:08.002062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:08.002062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:08.002062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:08.002932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:08.002932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:08.002932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:08.002932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:08.002932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:08.002932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:08.002932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:08.002932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:08.002932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:08.002932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:08.002932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:15:08.002932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:08.003991 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 214/418] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:15:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:08.009223 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed2c94-4e89-432c-8bf2-104dd0f9df39 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:08.075130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:08.163574 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed2c94-4e89-432c-8bf2-104dd0f9df39 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:08.217652 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ea4878-d975-419f-8ab8-8591a77bc47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:08.218205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea4878-d975-419f-8ab8-8591a77bc47a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:08.218490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea4878-d975-419f-8ab8-8591a77bc47a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:08.218764 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ea4878-d975-419f-8ab8-8591a77bc47a None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:08.219490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea4878-d975-419f-8ab8-8591a77bc47a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:08.219793 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea4878-d975-419f-8ab8-8591a77bc47a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:08.221175 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ea4878-d975-419f-8ab8-8591a77bc47a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:08.233266 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 205/419] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:08.238489 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f09c83-b4ba-44fb-84e5-0a9863362be0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:08.393308 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f09c83-b4ba-44fb-84e5-0a9863362be0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:08.438336 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd86ac-9059-47bb-852f-23d81f0c83eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:08.438963 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd86ac-9059-47bb-852f-23d81f0c83eb None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:08.439242 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd86ac-9059-47bb-852f-23d81f0c83eb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:08.439516 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd86ac-9059-47bb-852f-23d81f0c83eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:08.519933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:08.529395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd86ac-9059-47bb-852f-23d81f0c83eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:08.530934 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd86ac-9059-47bb-852f-23d81f0c83eb None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:08.590549 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde92703-415b-4fe5-ae09-05430fcd3396 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:08.590859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde92703-415b-4fe5-ae09-05430fcd3396 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:08.591102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde92703-415b-4fe5-ae09-05430fcd3396 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:08.592679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 206/420] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:08.597627 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56080d-b2ae-4e26-a78f-cf97c04df2ac None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:08.597942 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56080d-b2ae-4e26-a78f-cf97c04df2ac None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:08.598224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c56080d-b2ae-4e26-a78f-cf97c04df2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:08.619326 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4c56080d-b2ae-4e26-a78f-cf97c04df2ac None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:08.620729 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4c56080d-b2ae-4e26-a78f-cf97c04df2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:08.691079 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 215/421] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:15:08.696902 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd86ac-9059-47bb-852f-23d81f0c83eb None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:08.752472 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 207/422] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:08.784324 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56080d-b2ae-4e26-a78f-cf97c04df2ac None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:08.818057 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:08.824803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:08.833681 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd86ac-9059-47bb-852f-23d81f0c83eb None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:08.878192 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7389fb53-4a33-4ee0-8cb7-16218550ece5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:08.878835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389fb53-4a33-4ee0-8cb7-16218550ece5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:08.879472 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389fb53-4a33-4ee0-8cb7-16218550ece5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:08.879760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7389fb53-4a33-4ee0-8cb7-16218550ece5 None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:08.881200 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389fb53-4a33-4ee0-8cb7-16218550ece5 None admin] RBAC: Authorizing `identity:get_project(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:08.882998 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7389fb53-4a33-4ee0-8cb7-16218550ece5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:08.886084 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 216/423] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:15:08 2022] GET /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:08.887780 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd01346-3283-42fe-80d5-624b65b7f1cf None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:08.888070 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd01346-3283-42fe-80d5-624b65b7f1cf None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:08.888354 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd01346-3283-42fe-80d5-624b65b7f1cf None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:08.890479 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 217/424] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:08.891195 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd01346-3283-42fe-80d5-624b65b7f1cf None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:08.907001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c56080d-b2ae-4e26-a78f-cf97c04df2ac None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:08.945595 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cfa5c6-0ab3-4da6-99f9-b5a60b5411fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:08.946823 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfa5c6-0ab3-4da6-99f9-b5a60b5411fb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:08.947322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfa5c6-0ab3-4da6-99f9-b5a60b5411fb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:08.948375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfa5c6-0ab3-4da6-99f9-b5a60b5411fb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:08.966164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cfa5c6-0ab3-4da6-99f9-b5a60b5411fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:08.971402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-41cfa5c6-0ab3-4da6-99f9-b5a60b5411fb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:15:08.971402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:08.971402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:08.971402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:08.971402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:08.971402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:08.971402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:08.971402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:08.971402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:08.971402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:08.971402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:08.971402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:08.971402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:08.971402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:08.971402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:08.971402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:08.972289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:08.972289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:08.972289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:08.972289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:08.972289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:08.972289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:08.972289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:08.972289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:08.972289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:08.972289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:08.972289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:08.972289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:08.972289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:08.972289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:08.972289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:08.972289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:08.973107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:08.973107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:08.973107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:08.973107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:08.973107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:08.973107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:08.973107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:08.973107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:08.973107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:08.973107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:08.973107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:15:08.973107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:08.973107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 208/425] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:15:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:08.974396 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e88b5-2366-4dcb-b76b-64eb673ce676 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:08.974578 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e88b5-2366-4dcb-b76b-64eb673ce676 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:08.974864 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e88b5-2366-4dcb-b76b-64eb673ce676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:08.993948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-820e88b5-2366-4dcb-b76b-64eb673ce676 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:08.995268 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-820e88b5-2366-4dcb-b76b-64eb673ce676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:09.027693 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd01346-3283-42fe-80d5-624b65b7f1cf None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:09.065006 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcc8de-afcd-4e2e-984d-807ab85503b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:09.066017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcc8de-afcd-4e2e-984d-807ab85503b7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:09.066500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcc8de-afcd-4e2e-984d-807ab85503b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:09.066971 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcc8de-afcd-4e2e-984d-807ab85503b7 None admin] PATH_INFO: `/v3/groups/62ee23836ebe49419e6178e20ed8a2ec` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:09.068341 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcc8de-afcd-4e2e-984d-807ab85503b7 None admin] RBAC: Authorizing `identity:get_group(group_id=62ee23836ebe49419e6178e20ed8a2ec)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:09.068571 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 209/426] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:09.069485 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e88b5-2366-4dcb-b76b-64eb673ce676 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:09.070886 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fcc8de-afcd-4e2e-984d-807ab85503b7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:09.077925 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 218/427] 10.222.0.39 () {56 vars in 1180 bytes} [Thu Jul 28 22:15:08 2022] GET /identity/v3/groups/62ee23836ebe49419e6178e20ed8a2ec => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:09.079132 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcc8de-afcd-4e2e-984d-807ab85503b7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:09.096125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:09.116055 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:09.178711 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e88b5-2366-4dcb-b76b-64eb673ce676 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:09.193920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcc8de-afcd-4e2e-984d-807ab85503b7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:09.214449 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c9c66-de98-484c-9727-5fad206b0d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:09.215061 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c9c66-de98-484c-9727-5fad206b0d30 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:09.215391 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c9c66-de98-484c-9727-5fad206b0d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:09.215681 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c9c66-de98-484c-9727-5fad206b0d30 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:09.216407 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c9c66-de98-484c-9727-5fad206b0d30 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:09.216717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c9c66-de98-484c-9727-5fad206b0d30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:09.218140 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c9c66-de98-484c-9727-5fad206b0d30 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:09.227735 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 210/428] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:15:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:09.228413 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c9c66-de98-484c-9727-5fad206b0d30 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:09.233698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8421dd-75ed-4c57-ba56-e731f2dbdf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:09.234569 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8421dd-75ed-4c57-ba56-e731f2dbdf42 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:09.235048 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8421dd-75ed-4c57-ba56-e731f2dbdf42 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:09.235575 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8421dd-75ed-4c57-ba56-e731f2dbdf42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:09.236643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8421dd-75ed-4c57-ba56-e731f2dbdf42 None admin] RBAC: Adding query filter params (group.id=62ee23836ebe49419e6178e20ed8a2ec, role.id=4146966c23674402aebb9a5b21cf18ba, scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:09.237341 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8421dd-75ed-4c57-ba56-e731f2dbdf42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:09.239096 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8421dd-75ed-4c57-ba56-e731f2dbdf42 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:09.253565 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 219/429] 10.222.0.39 () {56 vars in 1376 bytes} [Thu Jul 28 22:15:09 2022] GET /identity/v3/role_assignments?group.id=62ee23836ebe49419e6178e20ed8a2ec&scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29&role.id=4146966c23674402aebb9a5b21cf18ba => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:09.255070 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8421dd-75ed-4c57-ba56-e731f2dbdf42 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:09.322031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:09.339443 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c9c66-de98-484c-9727-5fad206b0d30 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:09.410543 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8421dd-75ed-4c57-ba56-e731f2dbdf42 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:09.413201 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1d1ed-395d-4bff-a161-d42e25391608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:09.413732 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:09.413936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1d1ed-395d-4bff-a161-d42e25391608 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:09.414175 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1d1ed-395d-4bff-a161-d42e25391608 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:09.414453 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b1d1ed-395d-4bff-a161-d42e25391608 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:09.439733 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b1d1ed-395d-4bff-a161-d42e25391608 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:09.444900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-19b1d1ed-395d-4bff-a161-d42e25391608 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:15:09.444900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:09.444900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:09.444900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:09.444900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:09.444900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:09.444900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:09.444900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:09.444900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:09.444900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:09.444900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:09.444900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:09.444900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:09.444900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:09.444900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:09.444900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:09.445869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:09.445869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:09.445869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:09.445869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:09.445869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:09.445869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:09.445869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:09.445869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:09.445869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:09.445869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:09.445869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:09.445869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:09.445869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:09.445869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:09.445869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:09.445869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:09.446669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:09.446669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:09.446669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:09.446669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:09.446669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:09.446669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:09.446669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:09.446669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:09.446669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:09.446669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:09.446669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 28 22:15:09.446669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:09.446669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 211/430] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:15:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:09.454063 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f377baf-c41e-43b0-a3e6-1acd3850b8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:09.454712 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f377baf-c41e-43b0-a3e6-1acd3850b8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:09.454991 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f377baf-c41e-43b0-a3e6-1acd3850b8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:09.455260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f377baf-c41e-43b0-a3e6-1acd3850b8b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:09.455985 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1d1ed-395d-4bff-a161-d42e25391608 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:09.533602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f377baf-c41e-43b0-a3e6-1acd3850b8b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:09.535060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f377baf-c41e-43b0-a3e6-1acd3850b8b6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:09.606449 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b1d1ed-395d-4bff-a161-d42e25391608 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:09.637214 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:09.644281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 220/431] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:15:09.648693 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941c483-984f-4e0f-827e-6fd42da7c3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:09.649499 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941c483-984f-4e0f-827e-6fd42da7c3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:09.649650 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f377baf-c41e-43b0-a3e6-1acd3850b8b6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:09.650057 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941c483-984f-4e0f-827e-6fd42da7c3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:09.650496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941c483-984f-4e0f-827e-6fd42da7c3e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:09.651886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941c483-984f-4e0f-827e-6fd42da7c3e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:09.652336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941c483-984f-4e0f-827e-6fd42da7c3e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:09.653883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941c483-984f-4e0f-827e-6fd42da7c3e9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:09.665454 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 212/432] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:15:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:09.669215 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941c483-984f-4e0f-827e-6fd42da7c3e9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:09.680844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:09.783791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f377baf-c41e-43b0-a3e6-1acd3850b8b6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:09.801110 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941c483-984f-4e0f-827e-6fd42da7c3e9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:09.828229 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1b87b-2063-41c0-8f0c-a5a4c734d609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:09.829008 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1b87b-2063-41c0-8f0c-a5a4c734d609 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:09.829237 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1b87b-2063-41c0-8f0c-a5a4c734d609 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:09.829515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1b87b-2063-41c0-8f0c-a5a4c734d609 None admin] PATH_INFO: `/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:09.830843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1b87b-2063-41c0-8f0c-a5a4c734d609 None admin] RBAC: Authorizing `identity:get_project(project_id=530b68d3fa8e4d8a8f2073c0a0c62b29)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:09.832302 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1b87b-2063-41c0-8f0c-a5a4c734d609 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:09.834181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 221/433] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:15:09 2022] GET /identity/v3/projects/530b68d3fa8e4d8a8f2073c0a0c62b29 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:09.835203 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38389177-3312-412e-abf2-e925feeadcc2 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:09.837557 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38389177-3312-412e-abf2-e925feeadcc2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:09.837557 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38389177-3312-412e-abf2-e925feeadcc2 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:09.837557 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 222/434] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:09.838900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab16a0-9b64-4858-a379-dc3e459fcce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:09.839728 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab16a0-9b64-4858-a379-dc3e459fcce9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:09.840154 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab16a0-9b64-4858-a379-dc3e459fcce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:09.840556 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab16a0-9b64-4858-a379-dc3e459fcce9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:09.841216 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38389177-3312-412e-abf2-e925feeadcc2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:09.910830 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:09.914663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab16a0-9b64-4858-a379-dc3e459fcce9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:09.916119 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab16a0-9b64-4858-a379-dc3e459fcce9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:09.925141 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:09.961208 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38389177-3312-412e-abf2-e925feeadcc2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:10.004328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63605f3-40cb-4984-8440-0c2af53c7e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:10.005089 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63605f3-40cb-4984-8440-0c2af53c7e46 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:10.005257 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63605f3-40cb-4984-8440-0c2af53c7e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:10.005533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63605f3-40cb-4984-8440-0c2af53c7e46 None admin] PATH_INFO: `/v3/groups/62ee23836ebe49419e6178e20ed8a2ec` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:10.006703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63605f3-40cb-4984-8440-0c2af53c7e46 None admin] RBAC: Authorizing `identity:get_group(group_id=62ee23836ebe49419e6178e20ed8a2ec)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:10.008128 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63605f3-40cb-4984-8440-0c2af53c7e46 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:10.014471 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 223/435] 10.222.0.39 () {56 vars in 1180 bytes} [Thu Jul 28 22:15:09 2022] GET /identity/v3/groups/62ee23836ebe49419e6178e20ed8a2ec => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:10.014471 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f837339-9251-46f7-837a-e7357fa42505 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:10.014471 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f837339-9251-46f7-837a-e7357fa42505 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:10.014471 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f837339-9251-46f7-837a-e7357fa42505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:10.033872 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 213/436] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 22:15:10.033872 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab16a0-9b64-4858-a379-dc3e459fcce9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:10.035147 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9f837339-9251-46f7-837a-e7357fa42505 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:10.036464 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9f837339-9251-46f7-837a-e7357fa42505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:10.108934 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 224/437] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:10.109798 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f837339-9251-46f7-837a-e7357fa42505 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:10.136086 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab16a0-9b64-4858-a379-dc3e459fcce9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:10.144657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:10.177007 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:10.178744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea83c59-2f7b-4468-ab87-5c0f0a6d6ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:10.179452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea83c59-2f7b-4468-ab87-5c0f0a6d6ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:10.179730 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea83c59-2f7b-4468-ab87-5c0f0a6d6ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:10.180000 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea83c59-2f7b-4468-ab87-5c0f0a6d6ba3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:10.180778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea83c59-2f7b-4468-ab87-5c0f0a6d6ba3 None admin] RBAC: Adding query filter params (group.id=62ee23836ebe49419e6178e20ed8a2ec, role.id=4912a35790fe4d4297540156083d81d1, scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:10.181084 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea83c59-2f7b-4468-ab87-5c0f0a6d6ba3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:10.182711 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea83c59-2f7b-4468-ab87-5c0f0a6d6ba3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:10.188647 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 214/438] 10.222.0.39 () {56 vars in 1376 bytes} [Thu Jul 28 22:15:10 2022] GET /identity/v3/role_assignments?group.id=62ee23836ebe49419e6178e20ed8a2ec&scope.project.id=530b68d3fa8e4d8a8f2073c0a0c62b29&role.id=4912a35790fe4d4297540156083d81d1 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:10.198241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d1c30-3b59-4826-8179-0919d8885f70 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:10.198241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d1c30-3b59-4826-8179-0919d8885f70 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:10.198241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d1c30-3b59-4826-8179-0919d8885f70 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:10.198241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 215/439] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:10.198241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d1c30-3b59-4826-8179-0919d8885f70 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:10.234047 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f837339-9251-46f7-837a-e7357fa42505 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:10.273058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba931db9-0154-4e0f-8400-83750bf632e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:10.273775 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba931db9-0154-4e0f-8400-83750bf632e2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:10.274017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba931db9-0154-4e0f-8400-83750bf632e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:10.274293 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba931db9-0154-4e0f-8400-83750bf632e2 None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:10.275580 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba931db9-0154-4e0f-8400-83750bf632e2 None admin] RBAC: Authorizing `identity:get_project(project_id=e5b44df13345468b90906e747c09a977)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:10.278002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba931db9-0154-4e0f-8400-83750bf632e2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:10.279771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 225/440] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:15:10 2022] GET /identity/v3/projects/e5b44df13345468b90906e747c09a977 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:10.280962 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345222a-dfa2-43bf-bdc5-f7c3097063ca None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:10.281163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345222a-dfa2-43bf-bdc5-f7c3097063ca None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:10.284711 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b345222a-dfa2-43bf-bdc5-f7c3097063ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:10.304895 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b345222a-dfa2-43bf-bdc5-f7c3097063ca None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:10.306172 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b345222a-dfa2-43bf-bdc5-f7c3097063ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:10.326681 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d1c30-3b59-4826-8179-0919d8885f70 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:10.364602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996b783-2833-4141-baa9-3534ae3289ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:10.365244 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996b783-2833-4141-baa9-3534ae3289ee None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:10.365546 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996b783-2833-4141-baa9-3534ae3289ee None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:10.365829 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e996b783-2833-4141-baa9-3534ae3289ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:10.378306 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e996b783-2833-4141-baa9-3534ae3289ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:10.384242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-e996b783-2833-4141-baa9-3534ae3289ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:15:10.384242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:10.384242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:10.384242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:10.384242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:10.384242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:10.384242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:10.384242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:10.384242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:10.384242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:10.384242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:10.384242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:10.384242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:10.384242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:10.384242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:10.384242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:10.385462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:10.385462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:10.385462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:10.385462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:10.385462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:10.385462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:10.385462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:10.385462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:10.385462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:10.385462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:10.385462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:10.385462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:10.385462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:10.385462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:10.385462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:10.385462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:10.386340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:10.386340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:10.386340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:10.386340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:10.386340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:10.386340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:10.386340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:10.386340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:10.386340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:10.386340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:10.386340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:15:10.386340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:10.386970 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 226/441] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:10.386970 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345222a-dfa2-43bf-bdc5-f7c3097063ca None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:10.387102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 216/442] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:15:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:10.402660 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996b783-2833-4141-baa9-3534ae3289ee None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:10.435853 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:10.455794 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:10.508616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b345222a-dfa2-43bf-bdc5-f7c3097063ca None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:10.526148 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e996b783-2833-4141-baa9-3534ae3289ee None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:10.553845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba9bb8-0536-4bf3-981a-2ff94134ee01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:10.554071 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba9bb8-0536-4bf3-981a-2ff94134ee01 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:10.554277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba9bb8-0536-4bf3-981a-2ff94134ee01 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:10.555288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba9bb8-0536-4bf3-981a-2ff94134ee01 None admin] PATH_INFO: `/v3/groups/62ee23836ebe49419e6178e20ed8a2ec` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:10.556718 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba9bb8-0536-4bf3-981a-2ff94134ee01 None admin] RBAC: Authorizing `identity:get_group(group_id=62ee23836ebe49419e6178e20ed8a2ec)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:10.558551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbba9bb8-0536-4bf3-981a-2ff94134ee01 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:10.560731 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 227/443] 10.222.0.39 () {56 vars in 1180 bytes} [Thu Jul 28 22:15:10 2022] GET /identity/v3/groups/62ee23836ebe49419e6178e20ed8a2ec => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:10.560731 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba9bb8-0536-4bf3-981a-2ff94134ee01 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:10.569047 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f12cf-f559-47e1-8ec3-853d1559c4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:10.569757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f12cf-f559-47e1-8ec3-853d1559c4bd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:10.570150 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f12cf-f559-47e1-8ec3-853d1559c4bd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:10.570336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f12cf-f559-47e1-8ec3-853d1559c4bd None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:10.571036 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f12cf-f559-47e1-8ec3-853d1559c4bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:10.571341 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f12cf-f559-47e1-8ec3-853d1559c4bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:10.573353 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f12cf-f559-47e1-8ec3-853d1559c4bd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:10.590080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 217/444] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:10.590933 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f12cf-f559-47e1-8ec3-853d1559c4bd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:10.682851 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbba9bb8-0536-4bf3-981a-2ff94134ee01 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:10.708331 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:10.717642 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f12cf-f559-47e1-8ec3-853d1559c4bd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:10.718165 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:10.726926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585f408-a000-4641-be82-7e6b5944f5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:10.727564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585f408-a000-4641-be82-7e6b5944f5ae None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:10.727844 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585f408-a000-4641-be82-7e6b5944f5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:10.728117 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585f408-a000-4641-be82-7e6b5944f5ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:10.738146 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1585f408-a000-4641-be82-7e6b5944f5ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:10.743540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-1585f408-a000-4641-be82-7e6b5944f5ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:15:10.743540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:10.743540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:10.743540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:10.743540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:10.743540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:10.743540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:10.743540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:10.743540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:10.743540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:10.743540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:10.743540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:10.743540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:10.743540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:10.743540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:10.743540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:10.744920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:10.744920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:10.744920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:10.744920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:10.744920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:10.744920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:10.744920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:10.744920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:10.744920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:10.744920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:10.744920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:10.744920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:10.744920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:10.744920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:10.744920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:10.744920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:10.745995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:10.745995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:10.745995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:10.745995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:10.745995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:10.745995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:10.745995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:10.745995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:10.745995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:10.745995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:10.745995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:15:10.745995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:10.745995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 228/445] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:15:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:10.747036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585f408-a000-4641-be82-7e6b5944f5ae None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:10.755412 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f5cbd-fcd9-406c-86e6-dbbd08fcd7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:10.756021 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f5cbd-fcd9-406c-86e6-dbbd08fcd7e9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:10.756301 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f5cbd-fcd9-406c-86e6-dbbd08fcd7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:10.756576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397f5cbd-fcd9-406c-86e6-dbbd08fcd7e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:10.757361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f5cbd-fcd9-406c-86e6-dbbd08fcd7e9 None admin] RBAC: Adding query filter params (group.id=62ee23836ebe49419e6178e20ed8a2ec, role.id=4912a35790fe4d4297540156083d81d1, scope.project.id=e5b44df13345468b90906e747c09a977) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:10.757698 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f5cbd-fcd9-406c-86e6-dbbd08fcd7e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:10.759125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397f5cbd-fcd9-406c-86e6-dbbd08fcd7e9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:10.765199 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 218/446] 10.222.0.39 () {56 vars in 1376 bytes} [Thu Jul 28 22:15:10 2022] GET /identity/v3/role_assignments?group.id=62ee23836ebe49419e6178e20ed8a2ec&scope.project.id=e5b44df13345468b90906e747c09a977&role.id=4912a35790fe4d4297540156083d81d1 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:10.766058 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f5cbd-fcd9-406c-86e6-dbbd08fcd7e9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:10.866843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1585f408-a000-4641-be82-7e6b5944f5ae None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:10.882610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397f5cbd-fcd9-406c-86e6-dbbd08fcd7e9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:10.904047 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efaeedf-38a0-49c6-8599-2cdba9d89f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:10.904685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaeedf-38a0-49c6-8599-2cdba9d89f0a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:10.904971 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaeedf-38a0-49c6-8599-2cdba9d89f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:10.905248 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efaeedf-38a0-49c6-8599-2cdba9d89f0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:10.922690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93943c5-2b1c-4bc4-8096-4dcf959ed082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:10.923309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93943c5-2b1c-4bc4-8096-4dcf959ed082 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:10.923594 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93943c5-2b1c-4bc4-8096-4dcf959ed082 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:10.924618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93943c5-2b1c-4bc4-8096-4dcf959ed082 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:10.924618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93943c5-2b1c-4bc4-8096-4dcf959ed082 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:10.924885 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93943c5-2b1c-4bc4-8096-4dcf959ed082 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:10.926273 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93943c5-2b1c-4bc4-8096-4dcf959ed082 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:10.935424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 219/447] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:15:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:10.941217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93943c5-2b1c-4bc4-8096-4dcf959ed082 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:10.967850 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:10.974944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:10.993942 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efaeedf-38a0-49c6-8599-2cdba9d89f0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:10.995778 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efaeedf-38a0-49c6-8599-2cdba9d89f0a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:11.060337 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93943c5-2b1c-4bc4-8096-4dcf959ed082 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:11.099780 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acdb5d-7081-4360-b8d8-44ab3dfece2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:11.100409 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acdb5d-7081-4360-b8d8-44ab3dfece2f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:11.100696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acdb5d-7081-4360-b8d8-44ab3dfece2f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:11.100972 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6acdb5d-7081-4360-b8d8-44ab3dfece2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:11.104790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 229/448] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:15:11.110501 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efaeedf-38a0-49c6-8599-2cdba9d89f0a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:11.178296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6acdb5d-7081-4360-b8d8-44ab3dfece2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:11.179759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6acdb5d-7081-4360-b8d8-44ab3dfece2f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:11.198152 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:11.203516 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:11.216331 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efaeedf-38a0-49c6-8599-2cdba9d89f0a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:11.249605 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd45dc8-cb93-4f97-ab7b-68b8cb9b30d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:11.250182 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd45dc8-cb93-4f97-ab7b-68b8cb9b30d4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:11.250466 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd45dc8-cb93-4f97-ab7b-68b8cb9b30d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:11.250739 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd45dc8-cb93-4f97-ab7b-68b8cb9b30d4 None admin] PATH_INFO: `/v3/projects/e5b44df13345468b90906e747c09a977` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:11.251901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd45dc8-cb93-4f97-ab7b-68b8cb9b30d4 None admin] RBAC: Authorizing `identity:get_project(project_id=e5b44df13345468b90906e747c09a977)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:11.253344 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd45dc8-cb93-4f97-ab7b-68b8cb9b30d4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:11.255041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 230/449] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:15:11 2022] GET /identity/v3/projects/e5b44df13345468b90906e747c09a977 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:11.260402 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd45dc8-cb93-4f97-ab7b-68b8cb9b30d4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:11.283642 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 220/450] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 22:15:11.289240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acdb5d-7081-4360-b8d8-44ab3dfece2f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:11.362778 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd45dc8-cb93-4f97-ab7b-68b8cb9b30d4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:11.387149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6acdb5d-7081-4360-b8d8-44ab3dfece2f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:11.395651 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3013b-ab4c-4982-be13-c04fa5c7cb66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:11.396288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3013b-ab4c-4982-be13-c04fa5c7cb66 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:11.396571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3013b-ab4c-4982-be13-c04fa5c7cb66 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:11.396854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed3013b-ab4c-4982-be13-c04fa5c7cb66 None admin] PATH_INFO: `/v3/groups/62ee23836ebe49419e6178e20ed8a2ec` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:11.398116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3013b-ab4c-4982-be13-c04fa5c7cb66 None admin] RBAC: Authorizing `identity:get_group(group_id=62ee23836ebe49419e6178e20ed8a2ec)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:11.399557 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed3013b-ab4c-4982-be13-c04fa5c7cb66 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:11.406533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 231/451] 10.222.0.39 () {56 vars in 1180 bytes} [Thu Jul 28 22:15:11 2022] GET /identity/v3/groups/62ee23836ebe49419e6178e20ed8a2ec => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:11.406705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:11.407581 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3013b-ab4c-4982-be13-c04fa5c7cb66 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:11.422310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde89da-2a41-4eef-b730-d7622ef5ca31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:11.422944 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde89da-2a41-4eef-b730-d7622ef5ca31 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:11.423227 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde89da-2a41-4eef-b730-d7622ef5ca31 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:11.423505 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde89da-2a41-4eef-b730-d7622ef5ca31 None admin] PATH_INFO: `/v3/projects/26d9130b7556481bb766f1b06a63e1e5` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:11.424676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde89da-2a41-4eef-b730-d7622ef5ca31 None admin] RBAC: Authorizing `identity:get_project(project_id=26d9130b7556481bb766f1b06a63e1e5)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:11.426166 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde89da-2a41-4eef-b730-d7622ef5ca31 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:11.427814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 221/452] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:15:11 2022] GET /identity/v3/projects/26d9130b7556481bb766f1b06a63e1e5 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:11.433162 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde89da-2a41-4eef-b730-d7622ef5ca31 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:11.436224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:11.512058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed3013b-ab4c-4982-be13-c04fa5c7cb66 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:11.530278 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde89da-2a41-4eef-b730-d7622ef5ca31 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:11.545757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a784b1-c0f4-4887-8a8a-cebf61ccc79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:11.546375 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a784b1-c0f4-4887-8a8a-cebf61ccc79e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:11.546660 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a784b1-c0f4-4887-8a8a-cebf61ccc79e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:11.546941 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a784b1-c0f4-4887-8a8a-cebf61ccc79e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:11.547732 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a784b1-c0f4-4887-8a8a-cebf61ccc79e None admin] RBAC: Adding query filter params (group.id=62ee23836ebe49419e6178e20ed8a2ec, role.id=4146966c23674402aebb9a5b21cf18ba, scope.project.id=e5b44df13345468b90906e747c09a977) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:11.548048 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a784b1-c0f4-4887-8a8a-cebf61ccc79e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:11.549486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a784b1-c0f4-4887-8a8a-cebf61ccc79e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:11.555161 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 232/453] 10.222.0.39 () {56 vars in 1376 bytes} [Thu Jul 28 22:15:11 2022] GET /identity/v3/role_assignments?group.id=62ee23836ebe49419e6178e20ed8a2ec&scope.project.id=e5b44df13345468b90906e747c09a977&role.id=4146966c23674402aebb9a5b21cf18ba => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:11.563778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1f738-4bd1-4603-b5a3-0103a84dba0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:11.564448 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1f738-4bd1-4603-b5a3-0103a84dba0d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:11.564733 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1f738-4bd1-4603-b5a3-0103a84dba0d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:11.565013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1f738-4bd1-4603-b5a3-0103a84dba0d None admin] PATH_INFO: `/v3/groups/f8b9f199082546748b71117a9a5c9bb4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:11.566253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1f738-4bd1-4603-b5a3-0103a84dba0d None admin] RBAC: Authorizing `identity:get_group(group_id=f8b9f199082546748b71117a9a5c9bb4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:11.567743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f1f738-4bd1-4603-b5a3-0103a84dba0d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:11.569450 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 222/454] 10.222.0.39 () {56 vars in 1180 bytes} [Thu Jul 28 22:15:11 2022] GET /identity/v3/groups/f8b9f199082546748b71117a9a5c9bb4 => generated 229 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:11.576145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a784b1-c0f4-4887-8a8a-cebf61ccc79e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:11.669182 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:11.682174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a784b1-c0f4-4887-8a8a-cebf61ccc79e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:11.718758 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaf1a1f-b725-4903-bd41-c3c972682566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:11.719372 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf1a1f-b725-4903-bd41-c3c972682566 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:11.719656 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf1a1f-b725-4903-bd41-c3c972682566 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:11.719933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaf1a1f-b725-4903-bd41-c3c972682566 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:11.720751 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf1a1f-b725-4903-bd41-c3c972682566 None admin] RBAC: Adding query filter params (group.id=f8b9f199082546748b71117a9a5c9bb4, role.id=aded7bd43e974686b3c23bd9d4255b24, scope.project.id=26d9130b7556481bb766f1b06a63e1e5) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:11.721037 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf1a1f-b725-4903-bd41-c3c972682566 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:11.722532 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaf1a1f-b725-4903-bd41-c3c972682566 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:11.728564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 233/455] 10.222.0.39 () {56 vars in 1376 bytes} [Thu Jul 28 22:15:11 2022] GET /identity/v3/role_assignments?group.id=f8b9f199082546748b71117a9a5c9bb4&scope.project.id=26d9130b7556481bb766f1b06a63e1e5&role.id=aded7bd43e974686b3c23bd9d4255b24 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:14.103670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4637f4e-56c3-4e7f-8ff0-f9ce4c2982eb None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:14.104890 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4637f4e-56c3-4e7f-8ff0-f9ce4c2982eb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:14.105395 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4637f4e-56c3-4e7f-8ff0-f9ce4c2982eb None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:14.107264 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 223/456] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:14.112595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec8e01-b049-4623-8be3-2db59fd859e2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:14.113423 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec8e01-b049-4623-8be3-2db59fd859e2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:14.113942 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec8e01-b049-4623-8be3-2db59fd859e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:14.138597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-09ec8e01-b049-4623-8be3-2db59fd859e2 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:14.140076 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-09ec8e01-b049-4623-8be3-2db59fd859e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:14.240613 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 234/457] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:14.290173 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4637f4e-56c3-4e7f-8ff0-f9ce4c2982eb None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:14.372004 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:14.438845 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4637f4e-56c3-4e7f-8ff0-f9ce4c2982eb None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:14.439196 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bafd2-cab1-4ec7-b39c-c02621ffb17e None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:14.439492 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bafd2-cab1-4ec7-b39c-c02621ffb17e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:14.439772 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601bafd2-cab1-4ec7-b39c-c02621ffb17e None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:14.441429 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 235/458] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:15:14.446204 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7e356-a134-4091-9b55-25655a142761 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:14.446489 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7e356-a134-4091-9b55-25655a142761 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:14.446763 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d7e356-a134-4091-9b55-25655a142761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:14.485427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c1d7e356-a134-4091-9b55-25655a142761 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:14.486931 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c1d7e356-a134-4091-9b55-25655a142761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:14.495299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51aace0-ba5c-4443-aa43-1213457d8f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:14.496022 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aace0-ba5c-4443-aa43-1213457d8f0f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:14.496377 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aace0-ba5c-4443-aa43-1213457d8f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:14.496721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51aace0-ba5c-4443-aa43-1213457d8f0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:14.507869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51aace0-ba5c-4443-aa43-1213457d8f0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:14.509440 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51aace0-ba5c-4443-aa43-1213457d8f0f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:14.525102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-d51aace0-ba5c-4443-aa43-1213457d8f0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:14.525102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:14.525102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:14.525102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:14.525102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:14.525102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:14.525102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:14.525102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:14.525102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:14.525102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:14.525102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:14.525102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:14.525102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:14.525102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:14.525102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:14.526548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:14.526548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:14.526548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:14.526548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:14.526548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:14.526548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:14.526548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:14.526548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:14.526548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:14.526548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:14.526548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:14.526548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:14.526548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:14.526548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:14.526548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:14.526548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:14.526548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:14.527423 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:14.527423 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:14.527423 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:14.527423 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:14.527657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 224/459] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:14.528849 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94f1d0-aa4a-479f-a049-17cc673199e8 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:14.529138 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94f1d0-aa4a-479f-a049-17cc673199e8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:14.529412 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94f1d0-aa4a-479f-a049-17cc673199e8 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:14.530693 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 225/460] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:14.535307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94f1d0-aa4a-479f-a049-17cc673199e8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:14.571502 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 236/461] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:14.573421 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249a17b-8335-4773-b3f7-c1239ea29414 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:14.573759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249a17b-8335-4773-b3f7-c1239ea29414 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:14.574033 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249a17b-8335-4773-b3f7-c1239ea29414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:14.604656 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1249a17b-8335-4773-b3f7-c1239ea29414 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:14.605875 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1249a17b-8335-4773-b3f7-c1239ea29414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:14.614268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:14.667495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94f1d0-aa4a-479f-a049-17cc673199e8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:14.675347 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:14.685747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 237/462] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:14.686392 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249a17b-8335-4773-b3f7-c1239ea29414 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:14.729952 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3809dda-7caf-454d-924d-c6c657df0d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:14.730593 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3809dda-7caf-454d-924d-c6c657df0d52 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:14.730875 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3809dda-7caf-454d-924d-c6c657df0d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:14.731146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3809dda-7caf-454d-924d-c6c657df0d52 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:14.731966 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3809dda-7caf-454d-924d-c6c657df0d52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:14.732275 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3809dda-7caf-454d-924d-c6c657df0d52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:14.733783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3809dda-7caf-454d-924d-c6c657df0d52 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:14.750576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 226/463] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:14.751481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3809dda-7caf-454d-924d-c6c657df0d52 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:14.793539 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249a17b-8335-4773-b3f7-c1239ea29414 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:14.828954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb824a-c4d6-4f4b-84c3-1717dda555e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:14.829650 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb824a-c4d6-4f4b-84c3-1717dda555e8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:14.829940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb824a-c4d6-4f4b-84c3-1717dda555e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:14.830217 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bb824a-c4d6-4f4b-84c3-1717dda555e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:14.841052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bb824a-c4d6-4f4b-84c3-1717dda555e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:14.842564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bb824a-c4d6-4f4b-84c3-1717dda555e8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:14.844384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:14.860933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-98bb824a-c4d6-4f4b-84c3-1717dda555e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:14.860933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:14.860933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:14.860933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:14.860933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:14.860933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:14.860933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:14.860933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:14.860933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:14.860933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:14.860933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:14.860933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:14.860933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:14.860933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:14.860933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:14.861954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:14.861954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:14.861954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:14.861954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:14.861954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:14.861954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:14.861954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:14.861954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:14.861954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:14.861954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:14.861954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:14.861954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:14.861954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:14.861954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:14.861954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:14.861954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:14.861954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:14.863473 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:14.863473 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:14.863473 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:14.863473 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:14.863473 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 238/464] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:14.865577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3809dda-7caf-454d-924d-c6c657df0d52 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:14.868471 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb824a-c4d6-4f4b-84c3-1717dda555e8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:14.901710 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed09f4-a3ab-43b3-9390-8fd4db0e0c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:14.902356 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed09f4-a3ab-43b3-9390-8fd4db0e0c1f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:14.902636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed09f4-a3ab-43b3-9390-8fd4db0e0c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:14.902909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed09f4-a3ab-43b3-9390-8fd4db0e0c1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:14.914366 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed09f4-a3ab-43b3-9390-8fd4db0e0c1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:14.915843 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed09f4-a3ab-43b3-9390-8fd4db0e0c1f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:14.932217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a9ed09f4-a3ab-43b3-9390-8fd4db0e0c1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:14.932217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:14.932217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:14.932217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:14.932217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:14.932217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:14.932217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:14.932217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:14.932217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:14.932217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:14.932217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:14.932217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:14.932217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:14.932217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:14.932217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:14.934836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:14.934836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:14.934836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:14.934836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:14.934836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:14.934836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:14.934836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:14.934836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:14.934836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:14.934836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:14.934836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:14.934836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:14.934836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:14.934836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:14.934836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:14.934836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:14.934836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:14.936879 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:14.936879 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:14.936879 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:14.936879 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:14.941615 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 227/465] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:14.941615 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed09f4-a3ab-43b3-9390-8fd4db0e0c1f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:14.960783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:14.975208 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bb824a-c4d6-4f4b-84c3-1717dda555e8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:15.010841 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42437c9-ff2d-48a5-b7de-d38bf650e3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:15.011465 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42437c9-ff2d-48a5-b7de-d38bf650e3ee None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:15.011750 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42437c9-ff2d-48a5-b7de-d38bf650e3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:15.012027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42437c9-ff2d-48a5-b7de-d38bf650e3ee None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:15.012712 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42437c9-ff2d-48a5-b7de-d38bf650e3ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:15.014312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42437c9-ff2d-48a5-b7de-d38bf650e3ee None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:15.042444 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 239/466] 10.222.0.39 () {60 vars in 1132 bytes} [Thu Jul 28 22:15:14 2022] POST /identity/v3/users => generated 246 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:15:15.043411 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42437c9-ff2d-48a5-b7de-d38bf650e3ee None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:15.058666 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed09f4-a3ab-43b3-9390-8fd4db0e0c1f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:15.114609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc8712-53be-4635-97b8-4367ba0a70f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:15.115385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc8712-53be-4635-97b8-4367ba0a70f2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:15.115581 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc8712-53be-4635-97b8-4367ba0a70f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:15.117605 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc8712-53be-4635-97b8-4367ba0a70f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:15.117605 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc8712-53be-4635-97b8-4367ba0a70f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:15.117605 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc8712-53be-4635-97b8-4367ba0a70f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:15.118352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc8712-53be-4635-97b8-4367ba0a70f2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:15.120859 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:15.144472 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 228/467] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:15.144472 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ade69-9dd9-44f6-b5d4-2e9896889184 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:15.144472 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ade69-9dd9-44f6-b5d4-2e9896889184 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:15.144472 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ade69-9dd9-44f6-b5d4-2e9896889184 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:15.144472 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 229/468] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:15.149681 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ade69-9dd9-44f6-b5d4-2e9896889184 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:15.194879 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42437c9-ff2d-48a5-b7de-d38bf650e3ee None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:15.241520 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399df0b4-5a02-448b-b8ff-3b328a4ff60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:15.242188 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399df0b4-5a02-448b-b8ff-3b328a4ff60c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:15.242473 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399df0b4-5a02-448b-b8ff-3b328a4ff60c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:15.242750 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399df0b4-5a02-448b-b8ff-3b328a4ff60c None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:15.243455 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399df0b4-5a02-448b-b8ff-3b328a4ff60c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:15.243767 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399df0b4-5a02-448b-b8ff-3b328a4ff60c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:15.245296 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399df0b4-5a02-448b-b8ff-3b328a4ff60c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:15.248756 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:15.270541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 240/469] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:15.270541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a0f76-ae52-4ae6-be3f-7be0be16bc45 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:15.270541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a0f76-ae52-4ae6-be3f-7be0be16bc45 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:15.270541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a0f76-ae52-4ae6-be3f-7be0be16bc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:15.281394 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ade69-9dd9-44f6-b5d4-2e9896889184 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:15.292564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8d2a0f76-ae52-4ae6-be3f-7be0be16bc45 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:15.294202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8d2a0f76-ae52-4ae6-be3f-7be0be16bc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:15.330019 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda5b05-b976-4cd7-a557-636c4aeab21e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:15.331212 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda5b05-b976-4cd7-a557-636c4aeab21e None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:15.331867 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda5b05-b976-4cd7-a557-636c4aeab21e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:15.332400 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda5b05-b976-4cd7-a557-636c4aeab21e None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:15.333286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda5b05-b976-4cd7-a557-636c4aeab21e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:15.335946 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda5b05-b976-4cd7-a557-636c4aeab21e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:15.363622 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 230/470] 10.222.0.39 () {60 vars in 1132 bytes} [Thu Jul 28 22:15:15 2022] POST /identity/v3/users => generated 248 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:15:15.364175 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda5b05-b976-4cd7-a557-636c4aeab21e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:15.380137 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 241/471] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:15.413727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a0f76-ae52-4ae6-be3f-7be0be16bc45 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:15.474056 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:15.485458 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda5b05-b976-4cd7-a557-636c4aeab21e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:15.524037 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce060c9c-01ca-459a-9523-062b91cc5ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:15.524787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce060c9c-01ca-459a-9523-062b91cc5ac8 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:15.525113 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce060c9c-01ca-459a-9523-062b91cc5ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:15.525948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce060c9c-01ca-459a-9523-062b91cc5ac8 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:15.526914 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce060c9c-01ca-459a-9523-062b91cc5ac8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:15.528155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce060c9c-01ca-459a-9523-062b91cc5ac8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:15.536592 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:15.546335 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a0f76-ae52-4ae6-be3f-7be0be16bc45 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:15.566606 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 231/472] 10.222.0.39 () {60 vars in 1132 bytes} [Thu Jul 28 22:15:15 2022] POST /identity/v3/users => generated 248 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:15:15.605848 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef8f075-afa6-440f-bb33-dcda5b13d45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:15.606874 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8f075-afa6-440f-bb33-dcda5b13d45e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:15.607434 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8f075-afa6-440f-bb33-dcda5b13d45e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:15.607910 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef8f075-afa6-440f-bb33-dcda5b13d45e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:15.630725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8f075-afa6-440f-bb33-dcda5b13d45e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:15.632412 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef8f075-afa6-440f-bb33-dcda5b13d45e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:15.652978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-5ef8f075-afa6-440f-bb33-dcda5b13d45e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:15.652978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:15.652978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:15.652978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:15.652978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:15.652978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:15.652978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:15.652978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:15.652978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:15.652978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:15.652978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:15.652978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:15.652978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:15.652978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:15.652978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:15.653909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:15.653909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:15.653909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:15.653909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:15.653909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:15.653909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:15.653909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:15.653909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:15.653909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:15.653909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:15.653909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:15.653909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:15.653909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:15.653909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:15.653909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:15.653909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:15.653909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:15.654791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:15.654791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:15.654791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:15.654791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:15.654791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 242/473] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:15:15.662987 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce060c9c-01ca-459a-9523-062b91cc5ac8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:15.694454 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb84f4b-0d73-4ff8-91d3-b853f7e97907 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:15.694762 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb84f4b-0d73-4ff8-91d3-b853f7e97907 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:15.695020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb84f4b-0d73-4ff8-91d3-b853f7e97907 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:15.696613 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 243/474] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:15.703022 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb37de-fd18-4a70-b37b-79f1ab982461 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:15.703315 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb37de-fd18-4a70-b37b-79f1ab982461 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:15.703574 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb37de-fd18-4a70-b37b-79f1ab982461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:15.725052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-11cb37de-fd18-4a70-b37b-79f1ab982461 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:15.726352 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-11cb37de-fd18-4a70-b37b-79f1ab982461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:15.796496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce060c9c-01ca-459a-9523-062b91cc5ac8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:15.818302 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 244/475] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:15:15.841142 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa570742-9adb-4c1b-ac62-e95f179160a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:15.841858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa570742-9adb-4c1b-ac62-e95f179160a0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:15.842142 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa570742-9adb-4c1b-ac62-e95f179160a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:15.842546 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa570742-9adb-4c1b-ac62-e95f179160a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:15.843249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa570742-9adb-4c1b-ac62-e95f179160a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:15.843556 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa570742-9adb-4c1b-ac62-e95f179160a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:15.844087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb37de-fd18-4a70-b37b-79f1ab982461 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:15.845001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa570742-9adb-4c1b-ac62-e95f179160a0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:15.862534 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 232/476] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:15.868206 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa570742-9adb-4c1b-ac62-e95f179160a0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:15.876769 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:15.926824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:15.971784 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cb37de-fd18-4a70-b37b-79f1ab982461 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:15.995146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa570742-9adb-4c1b-ac62-e95f179160a0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:16.018138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e994779-df11-4d0d-928d-7800971df782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:16.019194 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e994779-df11-4d0d-928d-7800971df782 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:16.019671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e994779-df11-4d0d-928d-7800971df782 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:16.020135 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e994779-df11-4d0d-928d-7800971df782 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:16.021073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e994779-df11-4d0d-928d-7800971df782 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:16.022773 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e994779-df11-4d0d-928d-7800971df782 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:16.036637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.sql.core [None req-4e994779-df11-4d0d-928d-7800971df782 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 22:15:16.036637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 22:15:16.036637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [parameters: {'id': '36cbfd123ddb46d388c80a50ba880b93', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 22:15:16.036637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80362) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 22:15:16.052566 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04803e33-d4bf-46f6-978d-b8e07ca7727e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:16.053194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04803e33-d4bf-46f6-978d-b8e07ca7727e None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:16.053477 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04803e33-d4bf-46f6-978d-b8e07ca7727e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:16.053786 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04803e33-d4bf-46f6-978d-b8e07ca7727e None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:16.054475 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04803e33-d4bf-46f6-978d-b8e07ca7727e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:16.055972 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04803e33-d4bf-46f6-978d-b8e07ca7727e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:16.065894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-4e994779-df11-4d0d-928d-7800971df782 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 22:15:16.065894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:16.065894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 22:15:16.065894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 22:15:16.065894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 22:15:16.065894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 22:15:16.065894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 22:15:16.065894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 22:15:16.065894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 22:15:16.065894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application conn.query(q) Jul 28 22:15:16.065894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 22:15:16.065894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 22:15:16.065894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 22:15:16.065894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application result.read() Jul 28 22:15:16.065894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 22:15:16.066809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 22:15:16.066809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 22:15:16.066809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 22:15:16.066809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 22:15:16.066809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 22:15:16.066809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 22:15:16.066809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 22:15:16.066809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 22:15:16.066809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:16.066809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 22:15:16.066809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:16.066809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:16.066809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 22:15:16.066809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 22:15:16.066809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 28 22:15:16.066809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return ref.to_dict() Jul 28 22:15:16.066809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 22:15:16.066809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application next(self.gen) Jul 28 22:15:16.067714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 22:15:16.067714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application yield resource Jul 28 22:15:16.067714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 22:15:16.067714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application next(self.gen) Jul 28 22:15:16.067714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 22:15:16.067714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 22:15:16.067714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 22:15:16.067714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 22:15:16.067714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 22:15:16.067714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise self.value Jul 28 22:15:16.067714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 22:15:16.067714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 22:15:16.067714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 22:15:16.067714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application session.commit() Jul 28 22:15:16.067714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 22:15:16.067714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 22:15:16.067714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 22:15:16.068564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 22:15:16.068564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 22:15:16.068564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self.session.flush() Jul 28 22:15:16.068564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 22:15:16.068564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 22:15:16.068564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 22:15:16.068564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 22:15:16.068564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 22:15:16.068564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application compat.raise_( Jul 28 22:15:16.068564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 22:15:16.068564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception Jul 28 22:15:16.068564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 22:15:16.068564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 22:15:16.068564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 22:15:16.068564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 22:15:16.068564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 22:15:16.068564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 22:15:16.069456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 22:15:16.069456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 22:15:16.069456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 22:15:16.069456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 22:15:16.069456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 22:15:16.069456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 22:15:16.069456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 22:15:16.069456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 22:15:16.069456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 22:15:16.069456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 22:15:16.069456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 22:15:16.069456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 22:15:16.069456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 22:15:16.069456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 22:15:16.069456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 22:15:16.069456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception Jul 28 22:15:16.073024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 22:15:16.073024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 22:15:16.073024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 22:15:16.073024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 22:15:16.073024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 22:15:16.073024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 22:15:16.073024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 22:15:16.073024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application conn.query(q) Jul 28 22:15:16.073024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 22:15:16.073024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 22:15:16.073024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 22:15:16.073024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application result.read() Jul 28 22:15:16.073024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 22:15:16.073024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 22:15:16.073024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 22:15:16.073024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 22:15:16.077288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 22:15:16.077288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 22:15:16.077288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 22:15:16.077288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 22:15:16.077288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 28 22:15:16.077288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 28 22:15:16.077288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [parameters: {'id': '36cbfd123ddb46d388c80a50ba880b93', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 28 22:15:16.077288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 22:15:16.077288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:16.077288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:15:16.077288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:16.077288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:16.077288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:16.077288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:16.077288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:16.077288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:16.078259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:16.078259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:16.078259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:16.078259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:16.078259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:16.078259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:16.078259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 28 22:15:16.078259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 28 22:15:16.078259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:16.078259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:16.078259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 28 22:15:16.078259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 28 22:15:16.078259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 22:15:16.078259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 22:15:16.078259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 28 22:15:16.078259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:16.079154 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 245/477] 10.222.0.39 () {60 vars in 1131 bytes} [Thu Jul 28 22:15:15 2022] POST /identity/v3/roles => generated 144 bytes in 224 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:16.080461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e994779-df11-4d0d-928d-7800971df782 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:16.116531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 233/478] 10.222.0.39 () {60 vars in 1132 bytes} [Thu Jul 28 22:15:15 2022] POST /identity/v3/users => generated 249 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:15:16.246954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:16.270205 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e994779-df11-4d0d-928d-7800971df782 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:16.331361 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c28cfd8-c692-4c08-ae98-93565b3113f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:16.332020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28cfd8-c692-4c08-ae98-93565b3113f6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:16.332301 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28cfd8-c692-4c08-ae98-93565b3113f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:16.332576 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c28cfd8-c692-4c08-ae98-93565b3113f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:16.344433 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c28cfd8-c692-4c08-ae98-93565b3113f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:16.349843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-3c28cfd8-c692-4c08-ae98-93565b3113f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:15:16.349843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:16.349843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:16.349843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:16.349843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:16.349843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:16.349843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:16.349843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:16.349843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:16.349843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:16.349843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:16.349843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:16.349843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:16.349843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:16.349843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:16.349843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:16.350717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:16.350717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:16.350717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:16.350717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:16.350717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:16.350717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:16.350717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:16.350717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:16.350717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:16.350717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:16.350717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:16.350717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:16.350717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:16.350717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:16.350717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:16.350717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:16.351500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:16.351500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:16.351500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:16.351500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:16.351500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:16.351500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:16.351500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:16.351500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:16.351500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:16.351500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:16.351500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 28 22:15:16.351500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:16.354795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 246/479] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:15:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:16.362622 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04803e33-d4bf-46f6-978d-b8e07ca7727e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:16.476804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04803e33-d4bf-46f6-978d-b8e07ca7727e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:16.484990 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:16.520245 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc35e38-a95c-4de7-9f98-fc116ef00106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:16.520813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc35e38-a95c-4de7-9f98-fc116ef00106 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:16.521100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc35e38-a95c-4de7-9f98-fc116ef00106 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:16.521378 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc35e38-a95c-4de7-9f98-fc116ef00106 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:16.522141 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc35e38-a95c-4de7-9f98-fc116ef00106 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:16.522450 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc35e38-a95c-4de7-9f98-fc116ef00106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:16.523841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc35e38-a95c-4de7-9f98-fc116ef00106 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:16.535289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 234/480] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:17.590186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf180b1-54fc-47f2-9266-b887df5cde0b None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:17.590670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf180b1-54fc-47f2-9266-b887df5cde0b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:17.590670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf180b1-54fc-47f2-9266-b887df5cde0b None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:17.592272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 247/481] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:17.597193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b515df-5ae7-4b95-a998-43d93061513e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:17.597641 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b515df-5ae7-4b95-a998-43d93061513e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:17.597829 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b515df-5ae7-4b95-a998-43d93061513e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:17.621720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-86b515df-5ae7-4b95-a998-43d93061513e None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:17.622925 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-86b515df-5ae7-4b95-a998-43d93061513e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:17.699645 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 235/482] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:17.735698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf180b1-54fc-47f2-9266-b887df5cde0b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:17.846047 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf180b1-54fc-47f2-9266-b887df5cde0b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:17.882231 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac88e109-ae60-40f6-96cf-df20a5676b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:17.882920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88e109-ae60-40f6-96cf-df20a5676b05 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:17.883204 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88e109-ae60-40f6-96cf-df20a5676b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:17.883532 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac88e109-ae60-40f6-96cf-df20a5676b05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:17.895072 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac88e109-ae60-40f6-96cf-df20a5676b05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:17.900312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ac88e109-ae60-40f6-96cf-df20a5676b05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:17.900312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:17.900312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:17.900312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:17.900312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:17.900312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:17.900312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:17.900312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:17.900312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:17.900312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:17.900312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:17.900312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:17.900312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:17.900312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:17.900312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:17.900312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:17.902996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:17.902996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:17.902996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:17.902996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:17.902996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:17.902996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:17.902996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:17.902996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:17.902996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:17.902996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:17.902996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:17.902996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:17.902996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:17.902996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:17.902996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:17.902996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:17.903987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:17.903987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:17.903987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:17.903987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:17.903987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:17.903987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:17.903987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:17.903987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:17.903987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:17.903987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:17.903987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:17.903987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:17.903987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 248/483] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:17.910850 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b515df-5ae7-4b95-a998-43d93061513e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:18.024664 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:18.033811 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b515df-5ae7-4b95-a998-43d93061513e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:18.078742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c01ba-e9eb-485d-b9dd-fb81fff17b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:18.079363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c01ba-e9eb-485d-b9dd-fb81fff17b65 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:18.079639 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c01ba-e9eb-485d-b9dd-fb81fff17b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:18.079906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48c01ba-e9eb-485d-b9dd-fb81fff17b65 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:18.080618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c01ba-e9eb-485d-b9dd-fb81fff17b65 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:18.080917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c01ba-e9eb-485d-b9dd-fb81fff17b65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:18.082298 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48c01ba-e9eb-485d-b9dd-fb81fff17b65 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:18.098139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e1b9b-2ce4-4971-86c7-cc8e7093be6f None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:18.098395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e1b9b-2ce4-4971-86c7-cc8e7093be6f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:18.098618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e1b9b-2ce4-4971-86c7-cc8e7093be6f None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:18.099941 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 249/484] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:15:18.100285 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 236/485] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:15:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:18.113754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c01ba-e9eb-485d-b9dd-fb81fff17b65 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:18.114012 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291175a-c688-4e5a-8682-ac296218408e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:18.114012 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291175a-c688-4e5a-8682-ac296218408e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:18.114012 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a291175a-c688-4e5a-8682-ac296218408e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:18.131950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:18.156932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a291175a-c688-4e5a-8682-ac296218408e None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:18.158368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a291175a-c688-4e5a-8682-ac296218408e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:18.225810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48c01ba-e9eb-485d-b9dd-fb81fff17b65 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:18.272997 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 250/486] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:15:18.278032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4755c-f990-400f-a9b3-7f218627d6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:18.278668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4755c-f990-400f-a9b3-7f218627d6fd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:18.278955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4755c-f990-400f-a9b3-7f218627d6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:18.279230 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4755c-f990-400f-a9b3-7f218627d6fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:18.290504 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4755c-f990-400f-a9b3-7f218627d6fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:18.292743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4755c-f990-400f-a9b3-7f218627d6fd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:18.302877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291175a-c688-4e5a-8682-ac296218408e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:18.308584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-12f4755c-f990-400f-a9b3-7f218627d6fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:18.308584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:18.308584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:18.308584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:18.308584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:18.308584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:18.308584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:18.308584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:18.308584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:18.308584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:18.308584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:18.308584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:18.308584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:18.308584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:18.308584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:18.309432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:18.309432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:18.309432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:18.309432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:18.309432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:18.309432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:18.309432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:18.309432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:18.309432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:18.309432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:18.309432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:18.309432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:18.309432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:18.309432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:18.309432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:18.309432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:18.309432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:18.310742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:18.310742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:18.310742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:18.310742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:18.310742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 237/487] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:18.316690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4755c-f990-400f-a9b3-7f218627d6fd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:18.339313 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:18.451883 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a291175a-c688-4e5a-8682-ac296218408e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:18.471017 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4755c-f990-400f-a9b3-7f218627d6fd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:18.497345 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc46498-a828-4c69-b0c1-0a11eda38265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:18.498123 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc46498-a828-4c69-b0c1-0a11eda38265 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:18.498471 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc46498-a828-4c69-b0c1-0a11eda38265 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:18.498817 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc46498-a828-4c69-b0c1-0a11eda38265 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:18.501960 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:18.512329 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc46498-a828-4c69-b0c1-0a11eda38265 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:18.514762 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5790282-df07-4f30-ae6b-98e9a1d8876b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:18.515590 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5790282-df07-4f30-ae6b-98e9a1d8876b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:18.516047 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5790282-df07-4f30-ae6b-98e9a1d8876b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:18.516518 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5790282-df07-4f30-ae6b-98e9a1d8876b None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:18.517394 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5790282-df07-4f30-ae6b-98e9a1d8876b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:18.517896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5790282-df07-4f30-ae6b-98e9a1d8876b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:18.519435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5790282-df07-4f30-ae6b-98e9a1d8876b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:18.519541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-5dc46498-a828-4c69-b0c1-0a11eda38265 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:18.519541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:18.519541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:18.519541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:18.519541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:18.519541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:18.519541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:18.519541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:18.519541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:18.519541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:18.519541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:18.519541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:18.519541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:18.519541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:18.519541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:18.519541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:18.520370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:18.520370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:18.520370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:18.520370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:18.520370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:18.520370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:18.520370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:18.520370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:18.520370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:18.520370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:18.520370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:18.520370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:18.520370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:18.520370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:18.520370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:18.520370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:18.521393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:18.521393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:18.521393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:18.521393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:18.521393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:18.521393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:18.521393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:18.521393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:18.521393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:18.521393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:18.521393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:18.521393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:18.521393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 251/488] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:15:18.522908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71204406-dcc0-4dad-b4d9-c5c94f01361d None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:18.523265 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71204406-dcc0-4dad-b4d9-c5c94f01361d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:18.523613 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71204406-dcc0-4dad-b4d9-c5c94f01361d None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:18.529990 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 252/489] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:18.529990 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71204406-dcc0-4dad-b4d9-c5c94f01361d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:18.541696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 238/490] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:18.542678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2eab6-55c9-4fd5-b009-6e6e7356d85d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:18.542967 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2eab6-55c9-4fd5-b009-6e6e7356d85d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:18.543277 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2eab6-55c9-4fd5-b009-6e6e7356d85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:18.564520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3bb2eab6-55c9-4fd5-b009-6e6e7356d85d None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:18.565762 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3bb2eab6-55c9-4fd5-b009-6e6e7356d85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:18.649790 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 239/491] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:18.651492 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2eab6-55c9-4fd5-b009-6e6e7356d85d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:18.657699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:18.671637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71204406-dcc0-4dad-b4d9-c5c94f01361d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:18.711698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e90362-59bd-4b43-adef-ce040db5aa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:18.712589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e90362-59bd-4b43-adef-ce040db5aa8a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:18.712589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e90362-59bd-4b43-adef-ce040db5aa8a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:18.713129 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e90362-59bd-4b43-adef-ce040db5aa8a None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:18.714040 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e90362-59bd-4b43-adef-ce040db5aa8a None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:18.714352 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e90362-59bd-4b43-adef-ce040db5aa8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:18.715752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e90362-59bd-4b43-adef-ce040db5aa8a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:18.725098 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 253/492] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:15:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:18.725947 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e90362-59bd-4b43-adef-ce040db5aa8a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:18.785303 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2eab6-55c9-4fd5-b009-6e6e7356d85d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:18.791712 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:18.826259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffdc033-77bb-445d-9793-42d62490890b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:18.827234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdc033-77bb-445d-9793-42d62490890b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:18.827693 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdc033-77bb-445d-9793-42d62490890b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:18.828170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffdc033-77bb-445d-9793-42d62490890b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:18.836656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffdc033-77bb-445d-9793-42d62490890b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:18.839557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffdc033-77bb-445d-9793-42d62490890b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:18.839779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e90362-59bd-4b43-adef-ce040db5aa8a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:18.854832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-cffdc033-77bb-445d-9793-42d62490890b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 22:15:18.854832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:18.854832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:18.854832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:18.854832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:18.854832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:18.854832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:18.854832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:18.854832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:18.854832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:18.854832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:18.854832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:18.854832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:18.854832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:18.854832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:18.854832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:18.855796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:18.855796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:18.855796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:18.855796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:18.855796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:18.855796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:18.855796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:18.855796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:18.855796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:18.855796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:18.855796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:18.855796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:18.855796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:18.855796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:18.855796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:18.855796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:18.855796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:18.855796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:18.856709 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:18.856709 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:18.856709 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:18.856709 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:18.856709 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:18.856709 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:18.856709 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:18.856709 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:18.856709 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 22:15:18.856709 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:18.858663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 240/493] 10.222.0.39 () {56 vars in 1093 bytes} [Thu Jul 28 22:15:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:18.859518 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffdc033-77bb-445d-9793-42d62490890b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:18.880780 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647340fd-d2a9-4ee7-95b4-2d0de8d2d5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:18.881424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647340fd-d2a9-4ee7-95b4-2d0de8d2d5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:18.881753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647340fd-d2a9-4ee7-95b4-2d0de8d2d5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:18.882034 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647340fd-d2a9-4ee7-95b4-2d0de8d2d5a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:18.898288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647340fd-d2a9-4ee7-95b4-2d0de8d2d5a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:18.904189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-647340fd-d2a9-4ee7-95b4-2d0de8d2d5a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:18.904189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:18.904189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:18.904189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:18.904189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:18.904189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:18.904189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:18.904189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:18.904189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:18.904189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:18.904189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:18.904189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:18.904189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:18.904189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:18.904189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:18.904189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:18.905136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:18.905136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:18.905136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:18.905136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:18.905136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:18.905136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:18.905136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:18.905136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:18.905136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:18.905136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:18.905136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:18.905136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:18.905136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:18.905136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:18.905136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:18.905136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:18.905993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:18.905993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:18.905993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:18.905993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:18.905993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:18.905993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:18.905993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:18.905993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:18.905993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:18.905993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:18.905993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:18.905993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:18.905993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 254/494] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:18.907167 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cb24e-d0a0-448e-8641-e843fc6032d5 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:18.907265 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cb24e-d0a0-448e-8641-e843fc6032d5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:18.907485 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3cb24e-d0a0-448e-8641-e843fc6032d5 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:18.917689 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 255/495] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:18 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:18.919699 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3cb24e-d0a0-448e-8641-e843fc6032d5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:18.935362 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:18.978440 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffdc033-77bb-445d-9793-42d62490890b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:19.013733 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a47fda-7055-4a2e-9f66-85e8974251b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:19.014384 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a47fda-7055-4a2e-9f66-85e8974251b4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:19.014934 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a47fda-7055-4a2e-9f66-85e8974251b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:19.015212 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a47fda-7055-4a2e-9f66-85e8974251b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:19.026699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a47fda-7055-4a2e-9f66-85e8974251b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:19.028182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a47fda-7055-4a2e-9f66-85e8974251b4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:19.036709 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3cb24e-d0a0-448e-8641-e843fc6032d5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:19.043496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-03a47fda-7055-4a2e-9f66-85e8974251b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:19.043496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:19.043496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:19.043496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:19.043496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:19.043496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:19.043496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:19.043496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:19.043496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:19.043496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:19.043496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:19.043496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:19.043496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:19.043496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:19.043496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:19.044376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:19.044376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:19.044376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:19.044376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:19.044376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:19.044376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:19.044376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:19.044376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:19.044376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:19.044376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:19.044376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:19.044376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:19.044376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:19.044376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:19.044376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:19.044376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:19.044376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:19.045219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:19.045219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:19.045219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:19.045219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:19.045219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 241/496] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:19.045490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a47fda-7055-4a2e-9f66-85e8974251b4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:19.064445 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:19.087946 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5673f-3c38-4cc8-9c73-1e5e7d3333f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:19.088709 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5673f-3c38-4cc8-9c73-1e5e7d3333f4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:19.089452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5673f-3c38-4cc8-9c73-1e5e7d3333f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:19.089764 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5673f-3c38-4cc8-9c73-1e5e7d3333f4 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:19.090582 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa5673f-3c38-4cc8-9c73-1e5e7d3333f4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:19.090894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa5673f-3c38-4cc8-9c73-1e5e7d3333f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:19.093018 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa5673f-3c38-4cc8-9c73-1e5e7d3333f4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:19.135047 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 256/497] 10.222.0.39 () {56 vars in 1133 bytes} [Thu Jul 28 22:15:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:19.136329 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b9398-fe1d-49c4-a06c-590b1f950659 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:19.136622 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b9398-fe1d-49c4-a06c-590b1f950659 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:19.136904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402b9398-fe1d-49c4-a06c-590b1f950659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:19.159377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-402b9398-fe1d-49c4-a06c-590b1f950659 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:19.160593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-402b9398-fe1d-49c4-a06c-590b1f950659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:19.164615 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a47fda-7055-4a2e-9f66-85e8974251b4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:19.180391 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:19.204184 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f9e97-2ed3-49aa-98dd-66f83b4ef15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:19.204776 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f9e97-2ed3-49aa-98dd-66f83b4ef15a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:19.205064 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f9e97-2ed3-49aa-98dd-66f83b4ef15a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:19.205339 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f9e97-2ed3-49aa-98dd-66f83b4ef15a None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:19.206094 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f9e97-2ed3-49aa-98dd-66f83b4ef15a None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:19.206549 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f9e97-2ed3-49aa-98dd-66f83b4ef15a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:19.207962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63f9e97-2ed3-49aa-98dd-66f83b4ef15a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:19.221095 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 242/498] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:15:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:19.221095 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f9e97-2ed3-49aa-98dd-66f83b4ef15a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:19.237193 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 257/499] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:19.237193 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b9398-fe1d-49c4-a06c-590b1f950659 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:19.336148 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f9e97-2ed3-49aa-98dd-66f83b4ef15a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:19.348818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402b9398-fe1d-49c4-a06c-590b1f950659 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:19.356697 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:19.370941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf32bf1-98ea-443e-877f-ac0da6843bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:19.371555 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf32bf1-98ea-443e-877f-ac0da6843bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:19.371941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf32bf1-98ea-443e-877f-ac0da6843bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:19.372216 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf32bf1-98ea-443e-877f-ac0da6843bc4 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:19.372908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf32bf1-98ea-443e-877f-ac0da6843bc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:19.373210 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf32bf1-98ea-443e-877f-ac0da6843bc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:19.374612 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf32bf1-98ea-443e-877f-ac0da6843bc4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:19.383444 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a0e53-5453-4265-a3b4-382f8c357ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:19.384062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a0e53-5453-4265-a3b4-382f8c357ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:19.384343 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a0e53-5453-4265-a3b4-382f8c357ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:19.384719 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a0e53-5453-4265-a3b4-382f8c357ee8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:19.386414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 243/500] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:19.387209 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf32bf1-98ea-443e-877f-ac0da6843bc4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:19.423060 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:19.453910 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a0e53-5453-4265-a3b4-382f8c357ee8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:19.455382 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a0e53-5453-4265-a3b4-382f8c357ee8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:19.487412 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf32bf1-98ea-443e-877f-ac0da6843bc4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:19.522477 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdfdc4-d689-441f-ba80-07ba38035443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:19.523307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdfdc4-d689-441f-ba80-07ba38035443 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:19.523583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdfdc4-d689-441f-ba80-07ba38035443 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:19.523853 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bdfdc4-d689-441f-ba80-07ba38035443 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:19.534355 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bdfdc4-d689-441f-ba80-07ba38035443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:19.535779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bdfdc4-d689-441f-ba80-07ba38035443 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:19.550692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-b2bdfdc4-d689-441f-ba80-07ba38035443 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:19.550692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:19.550692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:19.550692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:19.550692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:19.550692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:19.550692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:19.550692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:19.550692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:19.550692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:19.550692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:19.550692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:19.550692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:19.550692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:19.550692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:19.551701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:19.551701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:19.551701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:19.551701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:19.551701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:19.551701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:19.551701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:19.551701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:19.551701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:19.551701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:19.551701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:19.551701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:19.551701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:19.551701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:19.551701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:19.551701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:19.551701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:19.552559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:19.552559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:19.552559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:19.552559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:19.552559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 244/501] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:19.552840 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 258/502] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:15:19.552912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdfdc4-d689-441f-ba80-07ba38035443 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:19.553984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136721f3-e22e-4764-98f5-18f404119863 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:19.554272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136721f3-e22e-4764-98f5-18f404119863 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:19.554543 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136721f3-e22e-4764-98f5-18f404119863 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:19.555787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 259/503] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:19.557089 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136721f3-e22e-4764-98f5-18f404119863 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:19.567732 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:19.647603 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:19.660314 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bdfdc4-d689-441f-ba80-07ba38035443 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:19.661590 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136721f3-e22e-4764-98f5-18f404119863 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:19.693816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67408c-8ea0-4c35-a63b-6798e5f53e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:19.694451 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67408c-8ea0-4c35-a63b-6798e5f53e47 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:19.694732 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67408c-8ea0-4c35-a63b-6798e5f53e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:19.695003 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e67408c-8ea0-4c35-a63b-6798e5f53e47 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:19.695733 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2a5012-52a3-482c-8fd0-c73dbc65a63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:19.696360 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a5012-52a3-482c-8fd0-c73dbc65a63c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:19.696645 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a5012-52a3-482c-8fd0-c73dbc65a63c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:19.696913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2a5012-52a3-482c-8fd0-c73dbc65a63c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:19.704494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a5012-52a3-482c-8fd0-c73dbc65a63c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:19.705684 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e67408c-8ea0-4c35-a63b-6798e5f53e47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:19.705952 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2a5012-52a3-482c-8fd0-c73dbc65a63c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:19.710960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7e67408c-8ea0-4c35-a63b-6798e5f53e47 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:19.710960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:19.710960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:19.710960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:19.710960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:19.710960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:19.710960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:19.710960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:19.710960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:19.710960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:19.710960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:19.710960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:19.710960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:19.710960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:19.710960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:19.710960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:19.712028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:19.712028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:19.712028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:19.712028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:19.712028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:19.712028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:19.712028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:19.712028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:19.712028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:19.712028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:19.712028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:19.712028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:19.712028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:19.712028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:19.712028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:19.712028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:19.713099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:19.713099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:19.713099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:19.713099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:19.713099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:19.713099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:19.713099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:19.713099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:19.713099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:19.713099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:19.713099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:19.713099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:19.713099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 245/504] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:19.713099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67408c-8ea0-4c35-a63b-6798e5f53e47 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:19.723843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-fd2a5012-52a3-482c-8fd0-c73dbc65a63c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 22:15:19.723843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:19.723843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:19.723843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:19.723843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:19.723843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:19.723843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:19.723843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:19.723843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:19.723843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:19.723843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:19.723843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:19.723843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:19.723843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:19.723843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:19.723843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:19.724744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:19.724744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:19.724744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:19.724744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:19.724744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:19.724744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:19.724744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:19.724744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:19.724744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:19.724744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:19.724744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:19.724744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:19.724744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:19.724744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:19.724744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:19.724744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:19.724744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:19.724744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:19.725746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:19.725746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:19.725746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:19.725746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:19.725746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:19.725746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:19.725746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:19.725746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:19.725746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 22:15:19.725746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:19.725746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 260/505] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:15:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:19.726767 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9953fe4-7953-47fb-8324-ec78466de9e5 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:19.726921 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9953fe4-7953-47fb-8324-ec78466de9e5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:19.727244 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9953fe4-7953-47fb-8324-ec78466de9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:19.745868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c9953fe4-7953-47fb-8324-ec78466de9e5 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:19.747050 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c9953fe4-7953-47fb-8324-ec78466de9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:19.807112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:19.815479 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 261/506] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:19.815696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e67408c-8ea0-4c35-a63b-6798e5f53e47 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:19.816274 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9953fe4-7953-47fb-8324-ec78466de9e5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:19.857941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced8819-04a8-4e25-bddf-7bdb8b63636b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:19.858553 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced8819-04a8-4e25-bddf-7bdb8b63636b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:19.858830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced8819-04a8-4e25-bddf-7bdb8b63636b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:19.859102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced8819-04a8-4e25-bddf-7bdb8b63636b None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:19.859792 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced8819-04a8-4e25-bddf-7bdb8b63636b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:19.860094 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced8819-04a8-4e25-bddf-7bdb8b63636b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:19.861464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced8819-04a8-4e25-bddf-7bdb8b63636b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:19.873484 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 246/507] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:19.874285 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced8819-04a8-4e25-bddf-7bdb8b63636b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:19.884204 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:19.918360 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9953fe4-7953-47fb-8324-ec78466de9e5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:19.952085 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6bf56-7309-4110-99ad-7a2513c77361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:19.952730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6bf56-7309-4110-99ad-7a2513c77361 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:19.953012 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6bf56-7309-4110-99ad-7a2513c77361 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:19.953288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab6bf56-7309-4110-99ad-7a2513c77361 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:19.964002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6bf56-7309-4110-99ad-7a2513c77361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:19.965464 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab6bf56-7309-4110-99ad-7a2513c77361 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:19.973841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced8819-04a8-4e25-bddf-7bdb8b63636b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:19.980674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-5ab6bf56-7309-4110-99ad-7a2513c77361 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:19.980674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:19.980674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:19.980674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:19.980674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:19.980674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:19.980674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:19.980674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:19.980674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:19.980674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:19.980674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:19.980674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:19.980674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:19.980674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:19.980674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:19.981606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:19.981606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:19.981606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:19.981606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:19.981606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:19.981606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:19.981606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:19.981606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:19.981606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:19.981606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:19.981606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:19.981606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:19.981606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:19.981606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:19.981606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:19.981606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:19.981606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:19.982458 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:19.982458 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:19.982458 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:19.982458 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:19.982458 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 262/508] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:19.982734 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6bf56-7309-4110-99ad-7a2513c77361 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:20.011302 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c42895-b0d2-4111-bc47-d2f44dc3ccf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:20.011856 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c42895-b0d2-4111-bc47-d2f44dc3ccf1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:20.012142 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c42895-b0d2-4111-bc47-d2f44dc3ccf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:20.012415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c42895-b0d2-4111-bc47-d2f44dc3ccf1 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:20.013133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c42895-b0d2-4111-bc47-d2f44dc3ccf1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:20.013436 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c42895-b0d2-4111-bc47-d2f44dc3ccf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:20.015136 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c42895-b0d2-4111-bc47-d2f44dc3ccf1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:20.029191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 247/509] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:15:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:20.030037 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c42895-b0d2-4111-bc47-d2f44dc3ccf1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:20.032722 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:20.085245 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab6bf56-7309-4110-99ad-7a2513c77361 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:20.097589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:20.119180 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976e70e-e4e7-4321-93a7-b420da8455cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:20.119792 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976e70e-e4e7-4321-93a7-b420da8455cf None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:20.120073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976e70e-e4e7-4321-93a7-b420da8455cf None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:20.120349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4976e70e-e4e7-4321-93a7-b420da8455cf None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:20.121140 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976e70e-e4e7-4321-93a7-b420da8455cf None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:20.121445 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976e70e-e4e7-4321-93a7-b420da8455cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:20.122918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4976e70e-e4e7-4321-93a7-b420da8455cf None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:20.129337 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c42895-b0d2-4111-bc47-d2f44dc3ccf1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:20.140613 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 263/510] 10.222.0.39 () {56 vars in 1137 bytes} [Thu Jul 28 22:15:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:20.141769 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976e70e-e4e7-4321-93a7-b420da8455cf None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:20.168374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2ee13-14ca-478e-9860-3f709ada3419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:20.169010 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2ee13-14ca-478e-9860-3f709ada3419 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:20.169308 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2ee13-14ca-478e-9860-3f709ada3419 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:20.169618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea2ee13-14ca-478e-9860-3f709ada3419 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:20.179870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea2ee13-14ca-478e-9860-3f709ada3419 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:20.181970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea2ee13-14ca-478e-9860-3f709ada3419 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:20.195961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-bea2ee13-14ca-478e-9860-3f709ada3419 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:20.195961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:20.195961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:20.195961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:20.195961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:20.195961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:20.195961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:20.195961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:20.195961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:20.195961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:20.195961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:20.195961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:20.195961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:20.195961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:20.195961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:20.196827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:20.196827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:20.196827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:20.196827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:20.196827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:20.196827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:20.196827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:20.196827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:20.196827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:20.196827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:20.196827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:20.196827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:20.196827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:20.196827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:20.196827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:20.196827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:20.196827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:20.197770 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:20.197770 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:20.197770 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:20.197770 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:20.197770 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 248/511] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:20.198568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2ee13-14ca-478e-9860-3f709ada3419 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:20.246399 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4976e70e-e4e7-4321-93a7-b420da8455cf None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:20.253759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:20.280873 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a57813-b21f-47fa-a63e-997dd03233ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:20.281451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a57813-b21f-47fa-a63e-997dd03233ce None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:20.281768 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a57813-b21f-47fa-a63e-997dd03233ce None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:20.282050 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a57813-b21f-47fa-a63e-997dd03233ce None admin] PATH_INFO: `/v3/users/glance` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:20.289073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a57813-b21f-47fa-a63e-997dd03233ce None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:20.290547 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a57813-b21f-47fa-a63e-997dd03233ce None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:20.302625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-00a57813-b21f-47fa-a63e-997dd03233ce None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 22:15:20.302625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:20.302625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:20.302625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:20.302625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:20.302625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:20.302625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:20.302625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:20.302625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:20.302625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:20.302625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:20.302625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:20.302625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:20.302625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:20.302625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:20.302625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:20.303517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:20.303517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:20.303517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:20.303517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:20.303517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:20.303517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:20.303517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:20.303517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:20.303517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:20.303517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:20.303517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:20.303517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:20.303517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:20.303517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:20.303517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:20.303517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:20.303517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:20.303517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:20.304448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:20.304448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:20.304448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:20.304448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:20.304448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:20.304448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:20.304448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:20.304448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:20.304448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 22:15:20.304448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:20.304448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 264/512] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:15:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:20.305006 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a57813-b21f-47fa-a63e-997dd03233ce None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:20.312976 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea2ee13-14ca-478e-9860-3f709ada3419 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:20.335582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:20.348129 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a29cf-74b2-4bf7-ad44-a8df7c9d03fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:20.348747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a29cf-74b2-4bf7-ad44-a8df7c9d03fa None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:20.349031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a29cf-74b2-4bf7-ad44-a8df7c9d03fa None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:20.349310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389a29cf-74b2-4bf7-ad44-a8df7c9d03fa None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:20.350037 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a29cf-74b2-4bf7-ad44-a8df7c9d03fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:20.350343 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a29cf-74b2-4bf7-ad44-a8df7c9d03fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:20.351726 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389a29cf-74b2-4bf7-ad44-a8df7c9d03fa None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:20.363914 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 249/513] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:20.364701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a29cf-74b2-4bf7-ad44-a8df7c9d03fa None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:20.404465 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a57813-b21f-47fa-a63e-997dd03233ce None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:20.438425 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3838201-7ea3-442d-b81d-b8d210983072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:20.439069 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3838201-7ea3-442d-b81d-b8d210983072 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:20.439354 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3838201-7ea3-442d-b81d-b8d210983072 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:20.439632 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3838201-7ea3-442d-b81d-b8d210983072 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:20.450808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3838201-7ea3-442d-b81d-b8d210983072 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:20.452313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3838201-7ea3-442d-b81d-b8d210983072 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:20.467080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-c3838201-7ea3-442d-b81d-b8d210983072 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:20.467080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:20.467080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:20.467080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:20.467080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:20.467080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:20.467080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:20.467080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:20.467080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:20.467080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:20.467080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:20.467080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:20.467080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:20.467080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:20.467080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:20.467920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:20.467920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:20.467920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:20.467920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:20.467920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:20.467920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:20.467920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:20.467920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:20.467920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:20.467920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:20.467920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:20.467920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:20.467920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:20.467920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:20.467920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:20.467920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:20.467920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:20.468771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:20.468771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:20.468771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:20.468771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:20.468771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 265/514] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:20.469053 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389a29cf-74b2-4bf7-ad44-a8df7c9d03fa None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:20.470330 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3838201-7ea3-442d-b81d-b8d210983072 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:20.489156 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:20.503787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f133a-db16-48cd-8b15-798baa248348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:20.504392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f133a-db16-48cd-8b15-798baa248348 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:20.504675 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f133a-db16-48cd-8b15-798baa248348 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:20.504955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f133a-db16-48cd-8b15-798baa248348 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:20.555285 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:20.571764 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3838201-7ea3-442d-b81d-b8d210983072 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:20.575834 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f133a-db16-48cd-8b15-798baa248348 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:20.577342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f133a-db16-48cd-8b15-798baa248348 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:20.607256 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd5403-b1ce-4200-b8ce-7cbd0d940489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:20.607871 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd5403-b1ce-4200-b8ce-7cbd0d940489 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:20.608152 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd5403-b1ce-4200-b8ce-7cbd0d940489 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:20.608427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd5403-b1ce-4200-b8ce-7cbd0d940489 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:20.609124 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd5403-b1ce-4200-b8ce-7cbd0d940489 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:20.609426 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd5403-b1ce-4200-b8ce-7cbd0d940489 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:20.610830 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd5403-b1ce-4200-b8ce-7cbd0d940489 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:20.622953 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 266/515] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:20.626365 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd5403-b1ce-4200-b8ce-7cbd0d940489 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:20.677956 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 250/516] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:15:20.678790 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f133a-db16-48cd-8b15-798baa248348 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:20.704700 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:20.727740 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd5403-b1ce-4200-b8ce-7cbd0d940489 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:20.759056 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:20.761927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a223da-c332-4b34-aeb0-6d5922196064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:20.762548 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a223da-c332-4b34-aeb0-6d5922196064 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:20.762836 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a223da-c332-4b34-aeb0-6d5922196064 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:20.763110 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a223da-c332-4b34-aeb0-6d5922196064 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:20.763908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a223da-c332-4b34-aeb0-6d5922196064 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:20.764217 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a223da-c332-4b34-aeb0-6d5922196064 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:20.765931 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a223da-c332-4b34-aeb0-6d5922196064 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:20.779173 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f133a-db16-48cd-8b15-798baa248348 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:20.784292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 267/517] 10.222.0.39 () {56 vars in 1137 bytes} [Thu Jul 28 22:15:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:20.785106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a223da-c332-4b34-aeb0-6d5922196064 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:20.813019 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b207d-c95e-4f77-92ed-1e705e6e2269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:20.813641 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b207d-c95e-4f77-92ed-1e705e6e2269 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:20.813930 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b207d-c95e-4f77-92ed-1e705e6e2269 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:20.814208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8b207d-c95e-4f77-92ed-1e705e6e2269 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:20.820935 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b207d-c95e-4f77-92ed-1e705e6e2269 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:20.822955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8b207d-c95e-4f77-92ed-1e705e6e2269 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:20.835474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-9a8b207d-c95e-4f77-92ed-1e705e6e2269 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:20.835474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:20.835474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:20.835474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:20.835474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:20.835474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:20.835474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:20.835474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:20.835474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:20.835474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:20.835474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:20.835474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:20.835474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:20.835474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:20.835474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:20.836319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:20.836319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:20.836319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:20.836319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:20.836319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:20.836319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:20.836319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:20.836319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:20.836319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:20.836319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:20.836319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:20.836319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:20.836319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:20.836319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:20.836319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:20.836319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:20.836319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:20.837234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:20.837234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:20.837234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:20.837234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:20.837234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:20.837234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:20.837234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:20.837234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:20.837234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 251/518] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:20.838588 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b207d-c95e-4f77-92ed-1e705e6e2269 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:20.890216 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a223da-c332-4b34-aeb0-6d5922196064 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:20.924945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b113b-ae4c-4f57-8a4e-d785db2d824f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:20.925587 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b113b-ae4c-4f57-8a4e-d785db2d824f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:20.925884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b113b-ae4c-4f57-8a4e-d785db2d824f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:20.926160 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b113b-ae4c-4f57-8a4e-d785db2d824f None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:20.926859 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b113b-ae4c-4f57-8a4e-d785db2d824f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:20.927165 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b113b-ae4c-4f57-8a4e-d785db2d824f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:20.928549 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274b113b-ae4c-4f57-8a4e-d785db2d824f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:20.930938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:20.941019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 268/519] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:20.941805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b113b-ae4c-4f57-8a4e-d785db2d824f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:20.944816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8b207d-c95e-4f77-92ed-1e705e6e2269 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:20.979865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d5575-b92a-4889-8d3b-2691d8df664c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:20.980482 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d5575-b92a-4889-8d3b-2691d8df664c None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:20.980762 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d5575-b92a-4889-8d3b-2691d8df664c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:20.981033 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050d5575-b92a-4889-8d3b-2691d8df664c None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:20.981733 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d5575-b92a-4889-8d3b-2691d8df664c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:20.983161 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050d5575-b92a-4889-8d3b-2691d8df664c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:21.000156 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:21.006365 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 252/520] 10.222.0.39 () {60 vars in 1132 bytes} [Thu Jul 28 22:15:20 2022] POST /identity/v3/users => generated 247 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:15:21.007568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d5575-b92a-4889-8d3b-2691d8df664c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:21.062609 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b113b-ae4c-4f57-8a4e-d785db2d824f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:21.106687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2effcc6-a39c-407e-85e5-0a3b8ea2e55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:21.107495 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2effcc6-a39c-407e-85e5-0a3b8ea2e55f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:21.107784 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2effcc6-a39c-407e-85e5-0a3b8ea2e55f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:21.108060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2effcc6-a39c-407e-85e5-0a3b8ea2e55f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:21.120542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2effcc6-a39c-407e-85e5-0a3b8ea2e55f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:21.122179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2effcc6-a39c-407e-85e5-0a3b8ea2e55f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:21.126933 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050d5575-b92a-4889-8d3b-2691d8df664c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:21.139144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-d2effcc6-a39c-407e-85e5-0a3b8ea2e55f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:21.139144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:21.139144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:21.139144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:21.139144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:21.139144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:21.139144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:21.139144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:21.139144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:21.139144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:21.139144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:21.139144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:21.139144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:21.139144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:21.139144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:21.140467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:21.140467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:21.140467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:21.140467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:21.140467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:21.140467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:21.140467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:21.140467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:21.140467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:21.140467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:21.140467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:21.140467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:21.140467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:21.140467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:21.140467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:21.140467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:21.140467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:21.141328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:21.141328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:21.141328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:21.141328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:21.144586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 269/521] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:21.145862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2effcc6-a39c-407e-85e5-0a3b8ea2e55f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:21.175220 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d01ae9-c55a-41b1-af54-e83a3f70abe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:21.175842 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d01ae9-c55a-41b1-af54-e83a3f70abe3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:21.176128 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d01ae9-c55a-41b1-af54-e83a3f70abe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:21.176399 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d01ae9-c55a-41b1-af54-e83a3f70abe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:21.202710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:21.252884 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:21.256980 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d01ae9-c55a-41b1-af54-e83a3f70abe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:21.258603 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d01ae9-c55a-41b1-af54-e83a3f70abe3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:21.263973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2effcc6-a39c-407e-85e5-0a3b8ea2e55f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:21.300072 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466826e-dc54-4985-8a2f-a13d22de3151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:21.300691 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466826e-dc54-4985-8a2f-a13d22de3151 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:21.300979 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466826e-dc54-4985-8a2f-a13d22de3151 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:21.301264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2466826e-dc54-4985-8a2f-a13d22de3151 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:21.302053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466826e-dc54-4985-8a2f-a13d22de3151 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:21.302384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466826e-dc54-4985-8a2f-a13d22de3151 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:21.303824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2466826e-dc54-4985-8a2f-a13d22de3151 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:21.322828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 270/522] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:21.323665 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466826e-dc54-4985-8a2f-a13d22de3151 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:21.382582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 253/523] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:15:21.383767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d01ae9-c55a-41b1-af54-e83a3f70abe3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:21.429482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2466826e-dc54-4985-8a2f-a13d22de3151 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:21.440439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:21.469816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e8821-45e5-47d0-b0e9-111b0eda13f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:21.470476 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e8821-45e5-47d0-b0e9-111b0eda13f4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:21.470826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e8821-45e5-47d0-b0e9-111b0eda13f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:21.471113 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e8821-45e5-47d0-b0e9-111b0eda13f4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:21.478695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e8821-45e5-47d0-b0e9-111b0eda13f4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:21.480383 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e8821-45e5-47d0-b0e9-111b0eda13f4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:21.490675 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:21.492483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ad8e8821-45e5-47d0-b0e9-111b0eda13f4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 22:15:21.492483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:21.492483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:21.492483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:21.492483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:21.492483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:21.492483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:21.492483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:21.492483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:21.492483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:21.492483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:21.492483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:21.492483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:21.492483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:21.492483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:21.492483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:21.493668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:21.493668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:21.493668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:21.493668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:21.493668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:21.493668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:21.493668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:21.493668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:21.493668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:21.493668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:21.493668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:21.493668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:21.493668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:21.493668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:21.493668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:21.493668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:21.493668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:21.493668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:21.494797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:21.494797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:21.494797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:21.494797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:21.494797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:21.494797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:21.494797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:21.494797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:21.494797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 22:15:21.494797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:21.494797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 271/524] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:21.495812 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d01ae9-c55a-41b1-af54-e83a3f70abe3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:21.499625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e8821-45e5-47d0-b0e9-111b0eda13f4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:21.533046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008c688-1df5-4fd1-915f-a81f53f41a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:21.533704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008c688-1df5-4fd1-915f-a81f53f41a7e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:21.533995 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008c688-1df5-4fd1-915f-a81f53f41a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:21.534274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8008c688-1df5-4fd1-915f-a81f53f41a7e None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:21.534997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008c688-1df5-4fd1-915f-a81f53f41a7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:21.535301 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008c688-1df5-4fd1-915f-a81f53f41a7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:21.536666 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8008c688-1df5-4fd1-915f-a81f53f41a7e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:21.549135 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 254/525] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:21.549847 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008c688-1df5-4fd1-915f-a81f53f41a7e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:21.609868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e8821-45e5-47d0-b0e9-111b0eda13f4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:21.643076 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc5365-84d7-4dcc-924d-c59321816d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:21.643622 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc5365-84d7-4dcc-924d-c59321816d82 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:21.643918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc5365-84d7-4dcc-924d-c59321816d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:21.644181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc5365-84d7-4dcc-924d-c59321816d82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:21.644979 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc5365-84d7-4dcc-924d-c59321816d82 None admin] RBAC: Adding query filter params (role.id=a66ab77a63174592a1e468d2464e8bc4, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=77efe8046ea0468cb0c7c58c24fc4e75) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:21.645294 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc5365-84d7-4dcc-924d-c59321816d82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:21.646758 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cc5365-84d7-4dcc-924d-c59321816d82 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:21.652310 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 272/526] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:15:21 2022] GET /identity/v3/role_assignments?user.id=77efe8046ea0468cb0c7c58c24fc4e75&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=a66ab77a63174592a1e468d2464e8bc4 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:21.653112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc5365-84d7-4dcc-924d-c59321816d82 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:21.672595 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8008c688-1df5-4fd1-915f-a81f53f41a7e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:21.701728 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:21.715639 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217da140-302b-4152-9c29-ff1bdf9964a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:21.717074 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217da140-302b-4152-9c29-ff1bdf9964a9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:21.717619 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217da140-302b-4152-9c29-ff1bdf9964a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:21.718120 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217da140-302b-4152-9c29-ff1bdf9964a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:21.731716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217da140-302b-4152-9c29-ff1bdf9964a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:21.733748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217da140-302b-4152-9c29-ff1bdf9964a9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:21.751182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-217da140-302b-4152-9c29-ff1bdf9964a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:21.751182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:21.751182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:21.751182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:21.751182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:21.751182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:21.751182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:21.751182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:21.751182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:21.751182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:21.751182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:21.751182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:21.751182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:21.751182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:21.751182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:21.752146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:21.752146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:21.752146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:21.752146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:21.752146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:21.752146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:21.752146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:21.752146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:21.752146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:21.752146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:21.752146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:21.752146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:21.752146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:21.752146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:21.752146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:21.752146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:21.752146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:21.753005 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:21.753005 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:21.753005 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:21.753005 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:21.757190 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 255/527] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:21.758242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217da140-302b-4152-9c29-ff1bdf9964a9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:21.773853 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:21.799214 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc5365-84d7-4dcc-924d-c59321816d82 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:21.847153 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9423f4-7d6c-4281-91bb-60c7d26a2c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:21.848059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9423f4-7d6c-4281-91bb-60c7d26a2c12 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:21.848345 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9423f4-7d6c-4281-91bb-60c7d26a2c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:21.848627 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9423f4-7d6c-4281-91bb-60c7d26a2c12 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:21.849496 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9423f4-7d6c-4281-91bb-60c7d26a2c12 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:21.849840 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9423f4-7d6c-4281-91bb-60c7d26a2c12 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:21.851837 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9423f4-7d6c-4281-91bb-60c7d26a2c12 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:21.874968 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 273/528] 10.222.0.39 () {56 vars in 1139 bytes} [Thu Jul 28 22:15:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:21.875892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9423f4-7d6c-4281-91bb-60c7d26a2c12 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:21.883219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217da140-302b-4152-9c29-ff1bdf9964a9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:21.920443 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7ae41-b0cb-46d1-89a8-c09f2d78bf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:21.921035 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7ae41-b0cb-46d1-89a8-c09f2d78bf1f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:21.921320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7ae41-b0cb-46d1-89a8-c09f2d78bf1f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:21.921626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf7ae41-b0cb-46d1-89a8-c09f2d78bf1f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:21.930840 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7ae41-b0cb-46d1-89a8-c09f2d78bf1f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:21.933000 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf7ae41-b0cb-46d1-89a8-c09f2d78bf1f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:21.944752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ccf7ae41-b0cb-46d1-89a8-c09f2d78bf1f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:21.944752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:21.944752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:21.944752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:21.944752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:21.944752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:21.944752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:21.944752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:21.944752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:21.944752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:21.944752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:21.944752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:21.944752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:21.944752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:21.944752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:21.945662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:21.945662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:21.945662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:21.945662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:21.945662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:21.945662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:21.945662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:21.945662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:21.945662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:21.945662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:21.945662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:21.945662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:21.945662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:21.945662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:21.945662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:21.945662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:21.945662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:21.946542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:21.946542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:21.946542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:21.946542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:21.946542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:21.946542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:21.946542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:21.946542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:21.946542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 256/529] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:21.947318 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7ae41-b0cb-46d1-89a8-c09f2d78bf1f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:21.997890 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:22.002057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9423f4-7d6c-4281-91bb-60c7d26a2c12 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:22.020696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:22.047245 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6787e579-b8d7-4b0b-8514-33e815e1f4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:22.048238 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787e579-b8d7-4b0b-8514-33e815e1f4dd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:22.048718 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787e579-b8d7-4b0b-8514-33e815e1f4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:22.049231 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6787e579-b8d7-4b0b-8514-33e815e1f4dd None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:22.050147 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6787e579-b8d7-4b0b-8514-33e815e1f4dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:22.050607 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6787e579-b8d7-4b0b-8514-33e815e1f4dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:22.052264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6787e579-b8d7-4b0b-8514-33e815e1f4dd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:22.055422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf7ae41-b0cb-46d1-89a8-c09f2d78bf1f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:22.065377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 274/530] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:22.066401 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6787e579-b8d7-4b0b-8514-33e815e1f4dd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:22.096546 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e60f7-45b7-44bb-a86c-02c16fdf2058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:22.097048 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e60f7-45b7-44bb-a86c-02c16fdf2058 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:22.097441 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e60f7-45b7-44bb-a86c-02c16fdf2058 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:22.097853 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809e60f7-45b7-44bb-a86c-02c16fdf2058 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:22.186700 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6787e579-b8d7-4b0b-8514-33e815e1f4dd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:22.186902 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e60f7-45b7-44bb-a86c-02c16fdf2058 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:22.188663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809e60f7-45b7-44bb-a86c-02c16fdf2058 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:22.226251 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec0dd5-01a9-47f7-b7be-6e7ad12a031d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:22.227039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec0dd5-01a9-47f7-b7be-6e7ad12a031d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:22.227424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec0dd5-01a9-47f7-b7be-6e7ad12a031d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:22.227803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ec0dd5-01a9-47f7-b7be-6e7ad12a031d None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:22.228832 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec0dd5-01a9-47f7-b7be-6e7ad12a031d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:22.229247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec0dd5-01a9-47f7-b7be-6e7ad12a031d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:22.230866 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ec0dd5-01a9-47f7-b7be-6e7ad12a031d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:22.244905 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 275/531] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:22.245914 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec0dd5-01a9-47f7-b7be-6e7ad12a031d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:22.261724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:22.305877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:22.337566 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 257/532] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:15:22.337566 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e60f7-45b7-44bb-a86c-02c16fdf2058 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:22.371168 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ec0dd5-01a9-47f7-b7be-6e7ad12a031d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:22.414421 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1874ba4-6ab9-4d49-970a-67be3124b358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:22.415076 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1874ba4-6ab9-4d49-970a-67be3124b358 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:22.415360 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1874ba4-6ab9-4d49-970a-67be3124b358 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:22.415666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1874ba4-6ab9-4d49-970a-67be3124b358 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:22.425806 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1874ba4-6ab9-4d49-970a-67be3124b358 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:22.427234 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1874ba4-6ab9-4d49-970a-67be3124b358 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:22.440554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-c1874ba4-6ab9-4d49-970a-67be3124b358 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:22.440554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:22.440554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:22.440554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:22.440554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:22.440554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:22.440554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:22.440554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:22.440554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:22.440554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:22.440554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:22.440554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:22.440554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:22.440554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:22.440554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:22.441606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:22.441606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:22.441606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:22.441606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:22.441606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:22.441606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:22.441606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:22.441606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:22.441606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:22.441606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:22.441606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:22.441606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:22.441606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:22.441606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:22.441606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:22.441606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:22.441606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:22.442670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:22.442670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:22.442670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:22.442670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:22.442670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:22.442670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:22.442670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:22.442670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:22.442670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 276/533] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:22.442670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1874ba4-6ab9-4d49-970a-67be3124b358 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:22.470947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809e60f7-45b7-44bb-a86c-02c16fdf2058 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:22.508896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f069295-4c5e-4aa4-b49c-63be50986fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:22.510601 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f069295-4c5e-4aa4-b49c-63be50986fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:22.511087 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f069295-4c5e-4aa4-b49c-63be50986fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:22.511522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f069295-4c5e-4aa4-b49c-63be50986fa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:22.512644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f069295-4c5e-4aa4-b49c-63be50986fa3 None admin] RBAC: Adding query filter params (role.id=a66ab77a63174592a1e468d2464e8bc4, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=b4aa320b18ce43c3acfece4546fea0f1) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:22.513115 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f069295-4c5e-4aa4-b49c-63be50986fa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:22.515039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f069295-4c5e-4aa4-b49c-63be50986fa3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:22.520206 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:22.521969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 258/534] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:15:22 2022] GET /identity/v3/role_assignments?user.id=b4aa320b18ce43c3acfece4546fea0f1&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=a66ab77a63174592a1e468d2464e8bc4 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:22.522909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f069295-4c5e-4aa4-b49c-63be50986fa3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:22.556891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1874ba4-6ab9-4d49-970a-67be3124b358 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:22.564369 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:22.594674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efccceb-79f5-4f60-9f06-095bc1aa69cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:22.595384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efccceb-79f5-4f60-9f06-095bc1aa69cd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:22.595675 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efccceb-79f5-4f60-9f06-095bc1aa69cd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:22.596066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efccceb-79f5-4f60-9f06-095bc1aa69cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:22.607666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efccceb-79f5-4f60-9f06-095bc1aa69cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:22.609216 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efccceb-79f5-4f60-9f06-095bc1aa69cd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:22.625826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-0efccceb-79f5-4f60-9f06-095bc1aa69cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:22.625826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:22.625826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:22.625826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:22.625826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:22.625826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:22.625826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:22.625826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:22.625826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:22.625826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:22.625826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:22.625826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:22.625826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:22.625826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:22.625826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:22.626938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:22.626938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:22.626938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:22.626938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:22.626938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:22.626938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:22.626938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:22.626938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:22.626938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:22.626938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:22.626938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:22.626938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:22.626938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:22.626938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:22.626938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:22.626938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:22.626938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:22.628002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:22.628002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:22.628002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:22.628002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:22.631202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 277/535] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:22.633262 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efccceb-79f5-4f60-9f06-095bc1aa69cd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:22.649090 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f069295-4c5e-4aa4-b49c-63be50986fa3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:22.688315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e734a1-5004-44ef-a81a-04d110d619cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:22.689253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e734a1-5004-44ef-a81a-04d110d619cf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:22.689735 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e734a1-5004-44ef-a81a-04d110d619cf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:22.690157 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e734a1-5004-44ef-a81a-04d110d619cf None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:22.691146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e734a1-5004-44ef-a81a-04d110d619cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:22.691712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e734a1-5004-44ef-a81a-04d110d619cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:22.693404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e734a1-5004-44ef-a81a-04d110d619cf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:22.714470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 259/536] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:22.734660 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e734a1-5004-44ef-a81a-04d110d619cf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:22.757555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efccceb-79f5-4f60-9f06-095bc1aa69cd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:22.792529 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156755d-5ff0-40ea-9844-4abdc6114f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:22.793128 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156755d-5ff0-40ea-9844-4abdc6114f1d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:22.793415 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156755d-5ff0-40ea-9844-4abdc6114f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:22.803829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156755d-5ff0-40ea-9844-4abdc6114f1d None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:22.804587 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156755d-5ff0-40ea-9844-4abdc6114f1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:22.804908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156755d-5ff0-40ea-9844-4abdc6114f1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:22.813546 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156755d-5ff0-40ea-9844-4abdc6114f1d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:22.813546 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:22.824421 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 278/537] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:22.829567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156755d-5ff0-40ea-9844-4abdc6114f1d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:22.920896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:22.930336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e734a1-5004-44ef-a81a-04d110d619cf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:22.980550 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156755d-5ff0-40ea-9844-4abdc6114f1d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:22.995013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdaa87-8a98-4b4a-98ad-4ad3fcaf469b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:22.995832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdaa87-8a98-4b4a-98ad-4ad3fcaf469b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:22.996225 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdaa87-8a98-4b4a-98ad-4ad3fcaf469b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:22.996737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cdaa87-8a98-4b4a-98ad-4ad3fcaf469b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:22.998355 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cdaa87-8a98-4b4a-98ad-4ad3fcaf469b None admin] RBAC: Adding query filter params (role.id=a66ab77a63174592a1e468d2464e8bc4, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=0fecb440c0874ede9f3cf544cfed93ab) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:22.998910 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cdaa87-8a98-4b4a-98ad-4ad3fcaf469b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:23.001753 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cdaa87-8a98-4b4a-98ad-4ad3fcaf469b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:23.025616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 260/538] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:15:22 2022] GET /identity/v3/role_assignments?user.id=0fecb440c0874ede9f3cf544cfed93ab&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=a66ab77a63174592a1e468d2464e8bc4 => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:23.028259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45689c3f-a237-43a4-b54d-80fe624cd14c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:23.029113 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45689c3f-a237-43a4-b54d-80fe624cd14c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:23.029625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45689c3f-a237-43a4-b54d-80fe624cd14c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:23.030045 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45689c3f-a237-43a4-b54d-80fe624cd14c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:23.044263 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45689c3f-a237-43a4-b54d-80fe624cd14c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:23.046164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45689c3f-a237-43a4-b54d-80fe624cd14c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:23.065564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-45689c3f-a237-43a4-b54d-80fe624cd14c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:23.065564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:23.065564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:23.065564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:23.065564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:23.065564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:23.065564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:23.065564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:23.065564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:23.065564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:23.065564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:23.065564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:23.065564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:23.065564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:23.065564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:23.066488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:23.066488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:23.066488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:23.066488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:23.066488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:23.066488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:23.066488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:23.066488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:23.066488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:23.066488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:23.066488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:23.066488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:23.066488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:23.066488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:23.066488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:23.066488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:23.066488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:23.067735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:23.067735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:23.067735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:23.067735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:23.067735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:23.067735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:23.067735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:23.067735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:23.067735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 279/539] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:23.078726 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdaa87-8a98-4b4a-98ad-4ad3fcaf469b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:23.149275 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6155efa-6b34-4618-87ba-e942cd06e4db None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:23.149625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6155efa-6b34-4618-87ba-e942cd06e4db None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:23.149880 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6155efa-6b34-4618-87ba-e942cd06e4db None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:23.152220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 280/540] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:15:23.158984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df407353-8c04-4c0d-bec0-f5d68db832e8 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:23.159273 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df407353-8c04-4c0d-bec0-f5d68db832e8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:23.159533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df407353-8c04-4c0d-bec0-f5d68db832e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:23.196840 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-df407353-8c04-4c0d-bec0-f5d68db832e8 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:23.198127 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-df407353-8c04-4c0d-bec0-f5d68db832e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:23.209786 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cdaa87-8a98-4b4a-98ad-4ad3fcaf469b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:23.260996 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418e800-10f0-4811-8400-9c8cf0676b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:23.261543 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418e800-10f0-4811-8400-9c8cf0676b39 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:23.262260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418e800-10f0-4811-8400-9c8cf0676b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:23.262552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8418e800-10f0-4811-8400-9c8cf0676b39 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:23.264631 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8418e800-10f0-4811-8400-9c8cf0676b39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:23.264631 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8418e800-10f0-4811-8400-9c8cf0676b39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:23.266511 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8418e800-10f0-4811-8400-9c8cf0676b39 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:23.278058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:23.300951 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 261/541] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:23.307764 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418e800-10f0-4811-8400-9c8cf0676b39 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:23.336646 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 281/542] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:23.340630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:23.359652 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df407353-8c04-4c0d-bec0-f5d68db832e8 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:23.474321 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8418e800-10f0-4811-8400-9c8cf0676b39 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:23.523649 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca663f3f-e50c-4771-982e-090d5eaffccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:23.524202 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca663f3f-e50c-4771-982e-090d5eaffccb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:23.524478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca663f3f-e50c-4771-982e-090d5eaffccb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:23.524747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca663f3f-e50c-4771-982e-090d5eaffccb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:23.525865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca663f3f-e50c-4771-982e-090d5eaffccb None admin] RBAC: Adding query filter params (role.id=a66ab77a63174592a1e468d2464e8bc4, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=09b96431ac98447283960019cadde8a3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:23.526180 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca663f3f-e50c-4771-982e-090d5eaffccb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:23.527639 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca663f3f-e50c-4771-982e-090d5eaffccb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:23.540287 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 262/543] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:15:23 2022] GET /identity/v3/role_assignments?user.id=09b96431ac98447283960019cadde8a3&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=a66ab77a63174592a1e468d2464e8bc4 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:23.571131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df407353-8c04-4c0d-bec0-f5d68db832e8 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:23.619918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53c8aa-77d7-4321-841a-4a0d73920fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:23.620595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53c8aa-77d7-4321-841a-4a0d73920fbd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:23.620881 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53c8aa-77d7-4321-841a-4a0d73920fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:23.621156 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53c8aa-77d7-4321-841a-4a0d73920fbd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:23.632746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53c8aa-77d7-4321-841a-4a0d73920fbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:23.638908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-6e53c8aa-77d7-4321-841a-4a0d73920fbd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:23.638908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:23.638908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:23.638908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:23.638908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:23.638908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:23.638908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:23.638908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:23.638908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:23.638908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:23.638908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:23.638908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:23.638908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:23.638908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:23.638908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:23.638908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:23.640083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:23.640083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:23.640083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:23.640083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:23.640083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:23.640083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:23.640083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:23.640083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:23.640083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:23.640083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:23.640083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:23.640083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:23.640083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:23.640083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:23.640083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:23.640083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:23.640884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:23.640884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:23.640884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:23.640884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:23.640884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:23.640884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:23.640884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:23.640884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:23.640884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:23.640884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:23.640884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:23.640884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:23.640884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 282/544] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:15:23.651058 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca663f3f-e50c-4771-982e-090d5eaffccb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:23.752592 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:23.774365 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca663f3f-e50c-4771-982e-090d5eaffccb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:23.816063 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2a9cb-6fa1-4996-9e6c-bbe3b072a188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:23.816691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2a9cb-6fa1-4996-9e6c-bbe3b072a188 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:23.816970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2a9cb-6fa1-4996-9e6c-bbe3b072a188 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:23.817240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2a9cb-6fa1-4996-9e6c-bbe3b072a188 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:23.817988 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2a9cb-6fa1-4996-9e6c-bbe3b072a188 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:23.818290 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2a9cb-6fa1-4996-9e6c-bbe3b072a188 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:23.819671 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2a9cb-6fa1-4996-9e6c-bbe3b072a188 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:23.829445 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 263/545] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:15:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:23.835154 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53c8aa-77d7-4321-841a-4a0d73920fbd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:23.885574 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:23.961974 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53c8aa-77d7-4321-841a-4a0d73920fbd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:24.010362 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e4169-4a5d-4bfd-9dd7-adfbfe0c1efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:24.011407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e4169-4a5d-4bfd-9dd7-adfbfe0c1efa None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:24.011915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e4169-4a5d-4bfd-9dd7-adfbfe0c1efa None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:24.012388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e4169-4a5d-4bfd-9dd7-adfbfe0c1efa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:24.027015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e4169-4a5d-4bfd-9dd7-adfbfe0c1efa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:24.028736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e4169-4a5d-4bfd-9dd7-adfbfe0c1efa None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:24.054282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-1a5e4169-4a5d-4bfd-9dd7-adfbfe0c1efa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:24.054282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:24.054282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:24.054282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:24.054282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:24.054282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:24.054282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:24.054282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:24.054282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:24.054282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:24.054282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:24.054282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:24.054282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:24.054282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:24.054282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:24.055176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:24.055176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:24.055176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:24.055176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:24.055176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:24.055176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:24.055176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:24.055176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:24.055176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:24.055176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:24.055176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:24.055176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:24.055176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:24.055176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:24.055176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:24.055176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:24.055176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:24.056021 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:24.056021 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:24.056021 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:24.056021 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:24.058328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 283/546] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:24.063588 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2a9cb-6fa1-4996-9e6c-bbe3b072a188 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:24.093478 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caff1fb-f43b-4fbf-9eca-1995c3e6549a None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:24.093802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caff1fb-f43b-4fbf-9eca-1995c3e6549a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:24.094124 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7caff1fb-f43b-4fbf-9eca-1995c3e6549a None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:24.095494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 284/547] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:15:24.105715 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83b4ea-0640-48c3-849b-f0baabffbcc0 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:24.106004 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83b4ea-0640-48c3-849b-f0baabffbcc0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:24.106287 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83b4ea-0640-48c3-849b-f0baabffbcc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:24.143279 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9c83b4ea-0640-48c3-849b-f0baabffbcc0 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:24.143930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9c83b4ea-0640-48c3-849b-f0baabffbcc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:24.226454 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2a9cb-6fa1-4996-9e6c-bbe3b072a188 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:24.273620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 285/548] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:15:24.284462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73419e-6793-433a-9323-e8b8e04165cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:24.285367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73419e-6793-433a-9323-e8b8e04165cc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:24.285946 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73419e-6793-433a-9323-e8b8e04165cc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:24.286418 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc73419e-6793-433a-9323-e8b8e04165cc None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:24.287329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc73419e-6793-433a-9323-e8b8e04165cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:24.287835 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc73419e-6793-433a-9323-e8b8e04165cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:24.289486 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc73419e-6793-433a-9323-e8b8e04165cc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:24.303700 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 264/549] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:24.307903 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83b4ea-0640-48c3-849b-f0baabffbcc0 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:24.318601 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73419e-6793-433a-9323-e8b8e04165cc None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:24.325003 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:24.329086 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:24.424689 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83b4ea-0640-48c3-849b-f0baabffbcc0 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:24.491315 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27c748-cf1c-4215-8074-0f059685c425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:24.492603 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27c748-cf1c-4215-8074-0f059685c425 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:24.494631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27c748-cf1c-4215-8074-0f059685c425 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:24.495120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa27c748-cf1c-4215-8074-0f059685c425 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:24.501729 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc73419e-6793-433a-9323-e8b8e04165cc None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:24.524604 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa27c748-cf1c-4215-8074-0f059685c425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:24.529808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-aa27c748-cf1c-4215-8074-0f059685c425 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:24.529808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:24.529808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:24.529808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:24.529808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:24.529808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:24.529808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:24.529808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:24.529808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:24.529808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:24.529808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:24.529808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:24.529808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:24.529808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:24.529808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:24.529808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:24.531258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:24.531258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:24.531258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:24.531258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:24.531258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:24.531258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:24.531258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:24.531258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:24.531258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:24.531258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:24.531258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:24.531258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:24.531258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:24.531258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:24.531258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:24.531258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:24.532224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:24.532224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:24.532224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:24.532224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:24.532224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:24.532224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:24.532224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:24.532224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:24.532224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:24.532224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:24.532224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:24.532224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:24.538067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 286/550] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:15:24.545823 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27c748-cf1c-4215-8074-0f059685c425 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:24.562138 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab57099-53ba-429e-a382-ab97dd17b7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:24.563130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab57099-53ba-429e-a382-ab97dd17b7bd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:24.563673 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab57099-53ba-429e-a382-ab97dd17b7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:24.564173 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab57099-53ba-429e-a382-ab97dd17b7bd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:24.579476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab57099-53ba-429e-a382-ab97dd17b7bd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:24.583685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab57099-53ba-429e-a382-ab97dd17b7bd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:24.610069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-fab57099-53ba-429e-a382-ab97dd17b7bd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 22:15:24.610069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:24.610069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:24.610069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:24.610069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:24.610069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:24.610069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:24.610069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:24.610069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:24.610069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:24.610069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:24.610069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:24.610069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:24.610069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:24.610069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:24.610069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:24.611034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:24.611034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:24.611034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:24.611034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:24.611034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:24.611034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:24.611034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:24.611034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:24.611034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:24.611034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:24.611034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:24.611034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:24.611034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:24.611034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:24.611034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:24.611034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:24.611034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:24.611034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:24.612107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:24.612107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:24.612107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:24.612107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:24.612107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:24.612107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:24.612107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:24.612107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:24.612107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 22:15:24.612107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:24.619950 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 265/551] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:15:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:24.623754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab57099-53ba-429e-a382-ab97dd17b7bd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:24.684074 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:24.714667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa27c748-cf1c-4215-8074-0f059685c425 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:24.754872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16956f9a-f916-42ba-ad92-39bf576afbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:24.756238 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16956f9a-f916-42ba-ad92-39bf576afbc4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:24.756733 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16956f9a-f916-42ba-ad92-39bf576afbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:24.757249 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16956f9a-f916-42ba-ad92-39bf576afbc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:24.758367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16956f9a-f916-42ba-ad92-39bf576afbc4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:24.758987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16956f9a-f916-42ba-ad92-39bf576afbc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:24.760890 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16956f9a-f916-42ba-ad92-39bf576afbc4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:24.770798 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 287/552] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:15:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:24.775913 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:24.778141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16956f9a-f916-42ba-ad92-39bf576afbc4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:24.871165 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab57099-53ba-429e-a382-ab97dd17b7bd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:24.883502 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16956f9a-f916-42ba-ad92-39bf576afbc4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:24.928596 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb00cca-6dbe-4706-ad1e-9ec3110fbfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:24.928596 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb00cca-6dbe-4706-ad1e-9ec3110fbfa4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:24.928596 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb00cca-6dbe-4706-ad1e-9ec3110fbfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:24.928596 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb00cca-6dbe-4706-ad1e-9ec3110fbfa4 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:24.928973 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb00cca-6dbe-4706-ad1e-9ec3110fbfa4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:24.928973 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb00cca-6dbe-4706-ad1e-9ec3110fbfa4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:24.928973 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb00cca-6dbe-4706-ad1e-9ec3110fbfa4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:24.930171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a1e06-25cb-419b-938f-4c739e6da6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:24.931088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a1e06-25cb-419b-938f-4c739e6da6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:24.931781 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a1e06-25cb-419b-938f-4c739e6da6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:24.932270 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876a1e06-25cb-419b-938f-4c739e6da6e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:24.946007 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:24.952483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a1e06-25cb-419b-938f-4c739e6da6e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:24.954168 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876a1e06-25cb-419b-938f-4c739e6da6e3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:24.967799 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 266/553] 10.222.0.39 () {56 vars in 1135 bytes} [Thu Jul 28 22:15:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:24.972962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb00cca-6dbe-4706-ad1e-9ec3110fbfa4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:24.989377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-876a1e06-25cb-419b-938f-4c739e6da6e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:24.989377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:24.989377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:24.989377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:24.989377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:24.989377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:24.989377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:24.989377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:24.989377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:24.989377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:24.989377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:24.989377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:24.989377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:24.989377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:24.989377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:24.990461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:24.990461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:24.990461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:24.990461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:24.990461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:24.990461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:24.990461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:24.990461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:24.990461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:24.990461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:24.990461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:24.990461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:24.990461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:24.990461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:24.990461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:24.990461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:24.990461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:24.991339 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:24.991339 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:24.991339 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:24.991339 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:25.003306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 288/554] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:25.008766 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a1e06-25cb-419b-938f-4c739e6da6e3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:25.081973 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb00cca-6dbe-4706-ad1e-9ec3110fbfa4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:25.108914 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:25.121254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2951aea-cb8d-4c96-ac55-5639155a6290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:25.122237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2951aea-cb8d-4c96-ac55-5639155a6290 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:25.122550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2951aea-cb8d-4c96-ac55-5639155a6290 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:25.123167 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2951aea-cb8d-4c96-ac55-5639155a6290 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:25.157837 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876a1e06-25cb-419b-938f-4c739e6da6e3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:25.194666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d77c94-d193-40a9-9bf9-e27c084825cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:25.195556 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d77c94-d193-40a9-9bf9-e27c084825cd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:25.195986 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d77c94-d193-40a9-9bf9-e27c084825cd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:25.196399 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d77c94-d193-40a9-9bf9-e27c084825cd None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:25.197770 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d77c94-d193-40a9-9bf9-e27c084825cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:25.197928 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2951aea-cb8d-4c96-ac55-5639155a6290 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:25.198383 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d77c94-d193-40a9-9bf9-e27c084825cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:25.199694 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2951aea-cb8d-4c96-ac55-5639155a6290 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:25.200051 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d77c94-d193-40a9-9bf9-e27c084825cd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:25.213880 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 289/555] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:25.215368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e21fe5-9e50-47b6-82ea-643fac579d2d None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:25.215794 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e21fe5-9e50-47b6-82ea-643fac579d2d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:25.216345 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e21fe5-9e50-47b6-82ea-643fac579d2d None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:25.221910 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 290/556] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:25.222833 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e21fe5-9e50-47b6-82ea-643fac579d2d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:25.295947 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:25.310387 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 267/557] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:15:25.311866 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93786280-0df3-465d-96d0-cb2354b3fb8b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:25.312259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93786280-0df3-465d-96d0-cb2354b3fb8b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:25.312736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93786280-0df3-465d-96d0-cb2354b3fb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:25.330980 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-93786280-0df3-465d-96d0-cb2354b3fb8b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:25.332591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-93786280-0df3-465d-96d0-cb2354b3fb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:25.350304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:25.361794 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e21fe5-9e50-47b6-82ea-643fac579d2d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:25.400948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4521a14-ab14-46ac-bc66-8e64b1148d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:25.401519 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4521a14-ab14-46ac-bc66-8e64b1148d2e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:25.401845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4521a14-ab14-46ac-bc66-8e64b1148d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:25.402124 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4521a14-ab14-46ac-bc66-8e64b1148d2e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:25.413717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 268/558] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:25.413717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93786280-0df3-465d-96d0-cb2354b3fb8b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:25.420454 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4521a14-ab14-46ac-bc66-8e64b1148d2e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:25.422212 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4521a14-ab14-46ac-bc66-8e64b1148d2e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:25.436020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-b4521a14-ab14-46ac-bc66-8e64b1148d2e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 22:15:25.436020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:25.436020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:25.436020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:25.436020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:25.436020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:25.436020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:25.436020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:25.436020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:25.436020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:25.436020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:25.436020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:25.436020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:25.436020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:25.436020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:25.436020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:25.437537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:25.437537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:25.437537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:25.437537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:25.437537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:25.437537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:25.437537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:25.437537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:25.437537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:25.437537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:25.437537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:25.437537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:25.437537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:25.437537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:25.437537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:25.437537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:25.437537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:25.437537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:25.438642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:25.438642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:25.438642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:25.438642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:25.438642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:25.438642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:25.438642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:25.438642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:25.438642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 22:15:25.438642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:25.441768 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 291/559] 10.222.0.39 () {56 vars in 1093 bytes} [Thu Jul 28 22:15:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:25.443025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756a8a4-b6b5-4662-ad65-e362ee7cd233 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:25.443309 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756a8a4-b6b5-4662-ad65-e362ee7cd233 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:25.443593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756a8a4-b6b5-4662-ad65-e362ee7cd233 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:25.444861 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 292/560] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:25.445565 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756a8a4-b6b5-4662-ad65-e362ee7cd233 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:25.525940 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93786280-0df3-465d-96d0-cb2354b3fb8b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:25.551688 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756a8a4-b6b5-4662-ad65-e362ee7cd233 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:25.560883 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:25.562254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda24a68-0020-48bc-b063-dc175d74168c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:25.563059 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda24a68-0020-48bc-b063-dc175d74168c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:25.563495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda24a68-0020-48bc-b063-dc175d74168c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:25.563772 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda24a68-0020-48bc-b063-dc175d74168c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:25.574945 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda24a68-0020-48bc-b063-dc175d74168c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:25.576727 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda24a68-0020-48bc-b063-dc175d74168c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:25.595297 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99820b5-c184-40ec-bcfd-1ab54defb23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:25.595941 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99820b5-c184-40ec-bcfd-1ab54defb23f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:25.596229 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99820b5-c184-40ec-bcfd-1ab54defb23f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:25.596481 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99820b5-c184-40ec-bcfd-1ab54defb23f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:25.597696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-eda24a68-0020-48bc-b063-dc175d74168c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:25.597696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:25.597696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:25.597696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:25.597696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:25.597696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:25.597696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:25.597696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:25.597696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:25.597696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:25.597696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:25.597696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:25.597696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:25.597696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:25.597696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:25.598563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:25.598563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:25.598563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:25.598563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:25.598563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:25.598563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:25.598563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:25.598563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:25.598563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:25.598563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:25.598563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:25.598563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:25.598563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:25.598563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:25.598563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:25.598563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:25.598563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:25.599513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:25.599513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:25.599513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:25.599513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:25.599513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 269/561] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:25.600403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda24a68-0020-48bc-b063-dc175d74168c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:25.604649 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:25.613610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99820b5-c184-40ec-bcfd-1ab54defb23f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:25.618854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-a99820b5-c184-40ec-bcfd-1ab54defb23f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:25.618854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:25.618854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:25.618854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:25.618854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:25.618854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:25.618854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:25.618854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:25.618854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:25.618854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:25.618854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:25.618854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:25.618854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:25.618854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:25.618854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:25.618854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:25.619779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:25.619779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:25.619779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:25.619779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:25.619779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:25.619779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:25.619779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:25.619779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:25.619779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:25.619779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:25.619779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:25.619779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:25.619779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:25.619779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:25.619779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:25.619779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:25.620664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:25.620664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:25.620664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:25.620664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:25.620664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:25.620664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:25.620664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:25.620664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:25.620664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:25.620664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:25.620664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:25.620664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:25.620664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 293/562] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:25.622057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6f7d2-e87c-483a-a1b2-86a54c12b81d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:25.622057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6f7d2-e87c-483a-a1b2-86a54c12b81d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:25.622057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b6f7d2-e87c-483a-a1b2-86a54c12b81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:25.639844 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-27b6f7d2-e87c-483a-a1b2-86a54c12b81d None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:25.641037 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-27b6f7d2-e87c-483a-a1b2-86a54c12b81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:25.717639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 294/563] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:25.717639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6f7d2-e87c-483a-a1b2-86a54c12b81d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:25.723625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda24a68-0020-48bc-b063-dc175d74168c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:25.758626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563d35c-8d6c-4a77-bf2f-4f4020b03951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:25.759908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563d35c-8d6c-4a77-bf2f-4f4020b03951 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:25.760187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563d35c-8d6c-4a77-bf2f-4f4020b03951 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:25.760553 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f563d35c-8d6c-4a77-bf2f-4f4020b03951 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:25.761339 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563d35c-8d6c-4a77-bf2f-4f4020b03951 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:25.761670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563d35c-8d6c-4a77-bf2f-4f4020b03951 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:25.763101 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f563d35c-8d6c-4a77-bf2f-4f4020b03951 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:25.783371 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 270/564] 10.222.0.39 () {56 vars in 1133 bytes} [Thu Jul 28 22:15:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:25.784193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563d35c-8d6c-4a77-bf2f-4f4020b03951 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:25.825333 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:25.831416 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b6f7d2-e87c-483a-a1b2-86a54c12b81d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:25.851276 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:25.868100 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecf712-f7c7-4a0d-a57b-4955d9a125a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:25.869455 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecf712-f7c7-4a0d-a57b-4955d9a125a1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:25.869856 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecf712-f7c7-4a0d-a57b-4955d9a125a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:25.870146 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ecf712-f7c7-4a0d-a57b-4955d9a125a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:25.870855 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ecf712-f7c7-4a0d-a57b-4955d9a125a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:25.871164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ecf712-f7c7-4a0d-a57b-4955d9a125a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:25.872544 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ecf712-f7c7-4a0d-a57b-4955d9a125a1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:25.889112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 295/565] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:25.889112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecf712-f7c7-4a0d-a57b-4955d9a125a1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:25.898321 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f563d35c-8d6c-4a77-bf2f-4f4020b03951 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:25.936604 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae60e81-93eb-46a6-acb9-9d44eeeb7940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:25.937532 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae60e81-93eb-46a6-acb9-9d44eeeb7940 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:25.937833 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae60e81-93eb-46a6-acb9-9d44eeeb7940 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:25.938410 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae60e81-93eb-46a6-acb9-9d44eeeb7940 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:25.939202 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae60e81-93eb-46a6-acb9-9d44eeeb7940 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:25.939515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae60e81-93eb-46a6-acb9-9d44eeeb7940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:25.941314 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae60e81-93eb-46a6-acb9-9d44eeeb7940 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:25.951280 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 271/566] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:15:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:25.951896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae60e81-93eb-46a6-acb9-9d44eeeb7940 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:25.996326 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ecf712-f7c7-4a0d-a57b-4955d9a125a1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:26.033085 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d2a5c-cb50-4b7d-a4cc-7b45faf29ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:26.033755 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d2a5c-cb50-4b7d-a4cc-7b45faf29ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:26.034045 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d2a5c-cb50-4b7d-a4cc-7b45faf29ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:26.034320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743d2a5c-cb50-4b7d-a4cc-7b45faf29ea5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:26.045317 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743d2a5c-cb50-4b7d-a4cc-7b45faf29ea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:26.050761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-743d2a5c-cb50-4b7d-a4cc-7b45faf29ea5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:26.050761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:26.050761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:26.050761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:26.050761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:26.050761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:26.050761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:26.050761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:26.050761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:26.050761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:26.050761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:26.050761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:26.050761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:26.050761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:26.050761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:26.050761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:26.051862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:26.051862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:26.051862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:26.051862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:26.051862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:26.051862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:26.051862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:26.051862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:26.051862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:26.051862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:26.051862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:26.051862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:26.051862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:26.051862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:26.051862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:26.051862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:26.052924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:26.052924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:26.052924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:26.052924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:26.052924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:26.052924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:26.052924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:26.052924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:26.052924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:26.052924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:26.052924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:26.052924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:26.052924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 296/567] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:26.054360 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d2a5c-cb50-4b7d-a4cc-7b45faf29ea5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:26.065840 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae60e81-93eb-46a6-acb9-9d44eeeb7940 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:26.072378 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:26.089252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:26.100291 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef6b46-cf05-4507-bfd7-f2da3b4c79cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:26.100941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef6b46-cf05-4507-bfd7-f2da3b4c79cc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:26.101226 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef6b46-cf05-4507-bfd7-f2da3b4c79cc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:26.101509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef6b46-cf05-4507-bfd7-f2da3b4c79cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:26.112377 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef6b46-cf05-4507-bfd7-f2da3b4c79cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:26.113869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef6b46-cf05-4507-bfd7-f2da3b4c79cc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:26.128425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ddef6b46-cf05-4507-bfd7-f2da3b4c79cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:26.128425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:26.128425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:26.128425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:26.128425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:26.128425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:26.128425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:26.128425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:26.128425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:26.128425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:26.128425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:26.128425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:26.128425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:26.128425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:26.128425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:26.129520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:26.129520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:26.129520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:26.129520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:26.129520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:26.129520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:26.129520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:26.129520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:26.129520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:26.129520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:26.129520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:26.129520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:26.129520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:26.129520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:26.129520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:26.129520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:26.129520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:26.130674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:26.130674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:26.130674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:26.130674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:26.130674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 272/568] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:26.130674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef6b46-cf05-4507-bfd7-f2da3b4c79cc None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:26.162914 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743d2a5c-cb50-4b7d-a4cc-7b45faf29ea5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:26.196558 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef6a96-bf85-4c16-89e3-b3a5a32a6be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:26.197266 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef6a96-bf85-4c16-89e3-b3a5a32a6be1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:26.197651 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef6a96-bf85-4c16-89e3-b3a5a32a6be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:26.197948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef6a96-bf85-4c16-89e3-b3a5a32a6be1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:26.204542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef6a96-bf85-4c16-89e3-b3a5a32a6be1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:26.206046 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef6a96-bf85-4c16-89e3-b3a5a32a6be1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:26.217852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-b3ef6a96-bf85-4c16-89e3-b3a5a32a6be1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:26.217852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:26.217852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:26.217852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:26.217852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:26.217852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:26.217852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:26.217852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:26.217852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:26.217852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:26.217852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:26.217852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:26.217852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:26.217852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:26.217852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:26.218877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:26.218877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:26.218877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:26.218877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:26.218877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:26.218877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:26.218877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:26.218877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:26.218877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:26.218877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:26.218877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:26.218877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:26.218877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:26.218877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:26.218877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:26.218877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:26.218877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:26.219779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:26.219779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:26.219779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:26.219779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:26.219779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:26.219779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:26.219779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:26.219779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:26.219779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 297/569] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:26.220263 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e5463-6da2-4f22-bb49-534d60a6f771 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:26.220467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e5463-6da2-4f22-bb49-534d60a6f771 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:26.220698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e5463-6da2-4f22-bb49-534d60a6f771 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:26.221949 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 298/570] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:26.222643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e5463-6da2-4f22-bb49-534d60a6f771 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:26.237893 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef6b46-cf05-4507-bfd7-f2da3b4c79cc None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:26.271957 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de42bc-f858-4685-8875-9b912c6c44c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:26.272517 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de42bc-f858-4685-8875-9b912c6c44c8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:26.272827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de42bc-f858-4685-8875-9b912c6c44c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:26.273102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90de42bc-f858-4685-8875-9b912c6c44c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:26.284138 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de42bc-f858-4685-8875-9b912c6c44c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:26.285601 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90de42bc-f858-4685-8875-9b912c6c44c8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:26.299468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-90de42bc-f858-4685-8875-9b912c6c44c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:26.299468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:26.299468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:26.299468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:26.299468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:26.299468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:26.299468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:26.299468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:26.299468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:26.299468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:26.299468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:26.299468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:26.299468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:26.299468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:26.299468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:26.300567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:26.300567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:26.300567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:26.300567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:26.300567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:26.300567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:26.300567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:26.300567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:26.300567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:26.300567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:26.300567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:26.300567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:26.300567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:26.300567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:26.300567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:26.300567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:26.300567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:26.301464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:26.301464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:26.301464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:26.301464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:26.301464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 273/571] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:26.301464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de42bc-f858-4685-8875-9b912c6c44c8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:26.306692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:26.329836 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e5463-6da2-4f22-bb49-534d60a6f771 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:26.332563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:26.364987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ec96d-6ee3-4217-9c8f-df9aa986e3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:26.365631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ec96d-6ee3-4217-9c8f-df9aa986e3ad None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:26.365917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ec96d-6ee3-4217-9c8f-df9aa986e3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:26.366196 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ec96d-6ee3-4217-9c8f-df9aa986e3ad None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:26.366907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ec96d-6ee3-4217-9c8f-df9aa986e3ad None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:26.367211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ec96d-6ee3-4217-9c8f-df9aa986e3ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:26.368680 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ec96d-6ee3-4217-9c8f-df9aa986e3ad None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:26.376892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 299/572] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:15:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:26.377726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ec96d-6ee3-4217-9c8f-df9aa986e3ad None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:26.406625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90de42bc-f858-4685-8875-9b912c6c44c8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:26.439234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31ee7aa-6d89-4a39-b506-6ddec4edada7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:26.439845 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ee7aa-6d89-4a39-b506-6ddec4edada7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:26.440127 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ee7aa-6d89-4a39-b506-6ddec4edada7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:26.440396 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31ee7aa-6d89-4a39-b506-6ddec4edada7 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:26.441079 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ee7aa-6d89-4a39-b506-6ddec4edada7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:26.441379 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ee7aa-6d89-4a39-b506-6ddec4edada7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:26.442763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31ee7aa-6d89-4a39-b506-6ddec4edada7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:26.460683 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 274/573] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:26.461940 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929f73d-6b56-486f-a449-220b1418d1c2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:26.462234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929f73d-6b56-486f-a449-220b1418d1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:26.462995 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929f73d-6b56-486f-a449-220b1418d1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:26.481021 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e929f73d-6b56-486f-a449-220b1418d1c2 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:26.482254 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ec96d-6ee3-4217-9c8f-df9aa986e3ad None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:26.482363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e929f73d-6b56-486f-a449-220b1418d1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:26.515439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfc940-d55c-48fe-9f66-114782e328a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:26.516051 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfc940-d55c-48fe-9f66-114782e328a6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:26.516332 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfc940-d55c-48fe-9f66-114782e328a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:26.516608 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfc940-d55c-48fe-9f66-114782e328a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:26.517353 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfc940-d55c-48fe-9f66-114782e328a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:26.517697 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfc940-d55c-48fe-9f66-114782e328a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:26.519127 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfc940-d55c-48fe-9f66-114782e328a6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:26.529057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:26.532467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 300/574] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:26.533622 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfc940-d55c-48fe-9f66-114782e328a6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:26.551430 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 275/575] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:26.552266 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929f73d-6b56-486f-a449-220b1418d1c2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:26.570277 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:26.645019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfc940-d55c-48fe-9f66-114782e328a6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:26.654538 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929f73d-6b56-486f-a449-220b1418d1c2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:26.679519 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21f113-1fed-4b66-a970-a4d32bcfdf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:26.680124 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21f113-1fed-4b66-a970-a4d32bcfdf56 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:26.680404 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21f113-1fed-4b66-a970-a4d32bcfdf56 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:26.680675 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21f113-1fed-4b66-a970-a4d32bcfdf56 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:26.681472 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21f113-1fed-4b66-a970-a4d32bcfdf56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:26.681800 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21f113-1fed-4b66-a970-a4d32bcfdf56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:26.683169 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21f113-1fed-4b66-a970-a4d32bcfdf56 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:26.688799 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc243742-21e5-4a1a-b34f-b5f7974555f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:26.689436 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc243742-21e5-4a1a-b34f-b5f7974555f1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:26.689785 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc243742-21e5-4a1a-b34f-b5f7974555f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:26.690022 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc243742-21e5-4a1a-b34f-b5f7974555f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:26.694007 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 301/576] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:26.694783 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21f113-1fed-4b66-a970-a4d32bcfdf56 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:26.701458 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc243742-21e5-4a1a-b34f-b5f7974555f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:26.702940 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc243742-21e5-4a1a-b34f-b5f7974555f1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:26.718541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-dc243742-21e5-4a1a-b34f-b5f7974555f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:26.718541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:26.718541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:26.718541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:26.718541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:26.718541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:26.718541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:26.718541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:26.718541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:26.718541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:26.718541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:26.718541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:26.718541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:26.718541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:26.718541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:26.719474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:26.719474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:26.719474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:26.719474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:26.719474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:26.719474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:26.719474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:26.719474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:26.719474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:26.719474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:26.719474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:26.719474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:26.719474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:26.719474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:26.719474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:26.719474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:26.719474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:26.720397 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:26.720397 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:26.720397 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:26.720397 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:26.720397 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 276/577] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:26.720823 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc243742-21e5-4a1a-b34f-b5f7974555f1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:26.755717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:26.791854 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:26.798947 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21f113-1fed-4b66-a970-a4d32bcfdf56 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:26.822629 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc243742-21e5-4a1a-b34f-b5f7974555f1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:26.832253 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762a461-d122-4910-85a5-c5265e1411a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:26.832887 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762a461-d122-4910-85a5-c5265e1411a2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:26.833172 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762a461-d122-4910-85a5-c5265e1411a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:26.833449 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762a461-d122-4910-85a5-c5265e1411a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:26.840638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762a461-d122-4910-85a5-c5265e1411a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:26.842137 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762a461-d122-4910-85a5-c5265e1411a2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:26.853958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-e762a461-d122-4910-85a5-c5265e1411a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:26.853958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:26.853958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:26.853958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:26.853958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:26.853958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:26.853958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:26.853958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:26.853958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:26.853958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:26.853958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:26.853958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:26.853958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:26.853958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:26.853958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:26.854849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:26.854849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:26.854849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:26.854849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:26.854849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:26.854849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:26.854849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:26.854849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:26.854849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:26.854849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:26.854849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:26.854849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:26.854849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:26.854849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:26.854849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:26.854849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:26.854849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:26.855760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:26.855760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:26.855760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:26.855760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:26.855760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:26.855760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:26.855760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:26.855760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:26.855760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 302/578] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:26.856250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762a461-d122-4910-85a5-c5265e1411a2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:26.860917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9bce15-5496-4bba-972a-c26c20b81838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:26.861555 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9bce15-5496-4bba-972a-c26c20b81838 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:26.861842 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9bce15-5496-4bba-972a-c26c20b81838 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:26.862114 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9bce15-5496-4bba-972a-c26c20b81838 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:26.862898 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9bce15-5496-4bba-972a-c26c20b81838 None admin] RBAC: Adding query filter params (role.id=a66ab77a63174592a1e468d2464e8bc4, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=e3c9107872f147f6ab8698b67bb545af) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:26.863204 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9bce15-5496-4bba-972a-c26c20b81838 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:26.864623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9bce15-5496-4bba-972a-c26c20b81838 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:26.871229 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 277/579] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:15:26 2022] GET /identity/v3/role_assignments?user.id=e3c9107872f147f6ab8698b67bb545af&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=a66ab77a63174592a1e468d2464e8bc4 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:26.871926 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9bce15-5496-4bba-972a-c26c20b81838 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:26.986059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762a461-d122-4910-85a5-c5265e1411a2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:26.992185 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9bce15-5496-4bba-972a-c26c20b81838 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:27.011631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:27.028022 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fbac4-6b91-4110-a7d6-7ecae68e04bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:27.028692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fbac4-6b91-4110-a7d6-7ecae68e04bf None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:27.028979 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fbac4-6b91-4110-a7d6-7ecae68e04bf None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:27.029257 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85fbac4-6b91-4110-a7d6-7ecae68e04bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:27.037516 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03749ff4-cef0-4130-a4e0-bfb7c6238e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:27.037516 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03749ff4-cef0-4130-a4e0-bfb7c6238e91 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:27.037516 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03749ff4-cef0-4130-a4e0-bfb7c6238e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:27.037516 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03749ff4-cef0-4130-a4e0-bfb7c6238e91 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:27.037926 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:27.042952 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85fbac4-6b91-4110-a7d6-7ecae68e04bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:27.045318 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03749ff4-cef0-4130-a4e0-bfb7c6238e91 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:27.046878 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03749ff4-cef0-4130-a4e0-bfb7c6238e91 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:27.048311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-a85fbac4-6b91-4110-a7d6-7ecae68e04bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:27.048311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:27.048311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:27.048311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:27.048311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:27.048311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:27.048311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:27.048311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:27.048311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:27.048311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:27.048311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:27.048311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:27.048311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:27.048311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:27.048311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:27.048311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:27.049259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:27.049259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:27.049259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:27.049259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:27.049259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:27.049259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:27.049259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:27.049259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:27.049259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:27.049259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:27.049259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:27.049259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:27.049259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:27.049259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:27.049259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:27.049259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:27.050742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:27.050742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:27.050742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:27.050742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:27.050742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:27.050742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:27.050742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:27.050742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:27.050742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:27.050742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:27.050742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:27.050742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:27.050742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 303/580] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:27.050742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fbac4-6b91-4110-a7d6-7ecae68e04bf None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:27.065421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-03749ff4-cef0-4130-a4e0-bfb7c6238e91 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 22:15:27.065421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:27.065421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:27.065421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:27.065421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:27.065421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:27.065421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:27.065421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:27.065421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:27.065421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:27.065421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:27.065421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:27.065421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:27.065421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:27.065421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:27.065421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:27.066676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:27.066676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:27.066676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:27.066676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:27.066676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:27.066676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:27.066676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:27.066676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:27.066676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:27.066676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:27.066676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:27.066676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:27.066676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:27.066676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:27.066676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:27.066676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:27.066676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:27.066676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:27.068169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:27.068169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:27.068169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:27.068169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:27.068169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:27.068169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:27.068169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:27.068169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:27.068169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 22:15:27.068169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:27.068169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 278/581] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:15:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:27.068169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03749ff4-cef0-4130-a4e0-bfb7c6238e91 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:27.165691 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85fbac4-6b91-4110-a7d6-7ecae68e04bf None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:27.177654 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03749ff4-cef0-4130-a4e0-bfb7c6238e91 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:27.204009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c50c6f-f10d-495b-ba11-044ce204ac3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:27.204627 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c50c6f-f10d-495b-ba11-044ce204ac3e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:27.204911 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c50c6f-f10d-495b-ba11-044ce204ac3e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:27.205188 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c50c6f-f10d-495b-ba11-044ce204ac3e None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:27.205918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c50c6f-f10d-495b-ba11-044ce204ac3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:27.206222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c50c6f-f10d-495b-ba11-044ce204ac3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:27.207841 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c50c6f-f10d-495b-ba11-044ce204ac3e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:27.215156 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91706ba-be4d-48d4-9bcc-5a4d720b7cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:27.215913 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91706ba-be4d-48d4-9bcc-5a4d720b7cdb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:27.216196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91706ba-be4d-48d4-9bcc-5a4d720b7cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:27.216471 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91706ba-be4d-48d4-9bcc-5a4d720b7cdb None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:27.217255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91706ba-be4d-48d4-9bcc-5a4d720b7cdb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:27.217609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91706ba-be4d-48d4-9bcc-5a4d720b7cdb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:27.219037 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91706ba-be4d-48d4-9bcc-5a4d720b7cdb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:27.222708 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 304/582] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:27.223671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c50c6f-f10d-495b-ba11-044ce204ac3e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:27.242011 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 279/583] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:27.242011 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91706ba-be4d-48d4-9bcc-5a4d720b7cdb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:27.272047 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:27.303412 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:27.340231 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c50c6f-f10d-495b-ba11-044ce204ac3e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:27.351013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91706ba-be4d-48d4-9bcc-5a4d720b7cdb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:27.378134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36ff22-19f4-43b8-acaa-5b7638e2d871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:27.379003 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36ff22-19f4-43b8-acaa-5b7638e2d871 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:27.379439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36ff22-19f4-43b8-acaa-5b7638e2d871 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:27.379716 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a36ff22-19f4-43b8-acaa-5b7638e2d871 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:27.380431 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a36ff22-19f4-43b8-acaa-5b7638e2d871 None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:27.380733 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a36ff22-19f4-43b8-acaa-5b7638e2d871 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:27.382179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a36ff22-19f4-43b8-acaa-5b7638e2d871 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:27.386868 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677398d-4fb9-4395-92ac-70ea6ce65da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:27.387758 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677398d-4fb9-4395-92ac-70ea6ce65da3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:27.387902 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677398d-4fb9-4395-92ac-70ea6ce65da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:27.388084 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4677398d-4fb9-4395-92ac-70ea6ce65da3 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:27.388876 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677398d-4fb9-4395-92ac-70ea6ce65da3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:27.389200 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677398d-4fb9-4395-92ac-70ea6ce65da3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:27.390873 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4677398d-4fb9-4395-92ac-70ea6ce65da3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:27.391686 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 305/584] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:15:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:27.396688 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36ff22-19f4-43b8-acaa-5b7638e2d871 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:27.416330 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 280/585] 10.222.0.39 () {56 vars in 1137 bytes} [Thu Jul 28 22:15:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:27.416330 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677398d-4fb9-4395-92ac-70ea6ce65da3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:27.503464 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a36ff22-19f4-43b8-acaa-5b7638e2d871 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:27.505094 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:27.520535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4677398d-4fb9-4395-92ac-70ea6ce65da3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:27.535214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:27.543469 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da030b06-78ec-4691-a14e-cefced27a792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:27.544400 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da030b06-78ec-4691-a14e-cefced27a792 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:27.544400 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da030b06-78ec-4691-a14e-cefced27a792 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:27.544683 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da030b06-78ec-4691-a14e-cefced27a792 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:27.552773 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da030b06-78ec-4691-a14e-cefced27a792 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:27.554297 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da030b06-78ec-4691-a14e-cefced27a792 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:27.558922 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbace3e7-bfa0-48fe-93d7-bcbb4a917ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:27.559591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbace3e7-bfa0-48fe-93d7-bcbb4a917ecc None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:27.559867 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbace3e7-bfa0-48fe-93d7-bcbb4a917ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:27.560137 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbace3e7-bfa0-48fe-93d7-bcbb4a917ecc None admin] PATH_INFO: `/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/77efe8046ea0468cb0c7c58c24fc4e75/roles/a66ab77a63174592a1e468d2464e8bc4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:27.566240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-da030b06-78ec-4691-a14e-cefced27a792 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 22:15:27.566240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:27.566240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:27.566240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:27.566240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:27.566240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:27.566240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:27.566240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:27.566240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:27.566240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:27.566240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:27.566240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:27.566240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:27.566240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:27.566240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:27.566240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:27.567163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:27.567163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:27.567163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:27.567163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:27.567163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:27.567163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:27.567163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:27.567163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:27.567163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:27.567163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:27.567163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:27.567163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:27.567163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:27.567163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:27.567163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:27.567163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:27.567163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:27.567163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:27.568099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:27.568099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:27.568099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:27.568099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:27.568099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:27.568099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:27.568099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:27.568099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:27.568099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 22:15:27.568099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:27.568099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 306/586] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:15:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:27.568709 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da030b06-78ec-4691-a14e-cefced27a792 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:27.580951 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbace3e7-bfa0-48fe-93d7-bcbb4a917ecc None admin] RBAC: Authorizing `identity:create_grant(project_id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user_id=77efe8046ea0468cb0c7c58c24fc4e75, role_id=a66ab77a63174592a1e468d2464e8bc4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:27.582432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbace3e7-bfa0-48fe-93d7-bcbb4a917ecc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:27.596577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 281/587] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:15:27 2022] PUT /identity/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/77efe8046ea0468cb0c7c58c24fc4e75/roles/a66ab77a63174592a1e468d2464e8bc4 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:15:27.597380 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbace3e7-bfa0-48fe-93d7-bcbb4a917ecc None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:27.720340 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da030b06-78ec-4691-a14e-cefced27a792 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:27.727117 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbace3e7-bfa0-48fe-93d7-bcbb4a917ecc None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:27.771979 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07968f5d-d10e-450c-af17-cd45443b8813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:27.772730 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07968f5d-d10e-450c-af17-cd45443b8813 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:27.773208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07968f5d-d10e-450c-af17-cd45443b8813 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:27.773667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07968f5d-d10e-450c-af17-cd45443b8813 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:27.773880 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be83adb-ad5f-4b9d-a089-b36471a40f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:27.775314 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be83adb-ad5f-4b9d-a089-b36471a40f27 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:27.775933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be83adb-ad5f-4b9d-a089-b36471a40f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:27.776216 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be83adb-ad5f-4b9d-a089-b36471a40f27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:27.787520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07968f5d-d10e-450c-af17-cd45443b8813 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:27.788583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be83adb-ad5f-4b9d-a089-b36471a40f27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:27.789508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07968f5d-d10e-450c-af17-cd45443b8813 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:27.790092 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be83adb-ad5f-4b9d-a089-b36471a40f27 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:27.805743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-4be83adb-ad5f-4b9d-a089-b36471a40f27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:27.805743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:27.805743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:27.805743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:27.805743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:27.805743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:27.805743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:27.805743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:27.805743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:27.805743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:27.805743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:27.805743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:27.805743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:27.805743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:27.805743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:27.806692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:27.806692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:27.806692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:27.806692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:27.806692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:27.806692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:27.806692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:27.806692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:27.806692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:27.806692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:27.806692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:27.806692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:27.806692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:27.806692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:27.806692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:27.806692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:27.806692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:27.807593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:27.807593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:27.807593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:27.807593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:27.807593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 307/588] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:27.808299 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be83adb-ad5f-4b9d-a089-b36471a40f27 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:27.809787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-07968f5d-d10e-450c-af17-cd45443b8813 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:27.809787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:27.809787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:27.809787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:27.809787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:27.809787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:27.809787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:27.809787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:27.809787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:27.809787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:27.809787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:27.809787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:27.809787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:27.809787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:27.809787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:27.810968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:27.810968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:27.810968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:27.810968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:27.810968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:27.810968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:27.810968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:27.810968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:27.810968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:27.810968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:27.810968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:27.810968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:27.810968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:27.810968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:27.810968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:27.810968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:27.810968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:27.811860 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:27.811860 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:27.811860 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:27.811860 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:27.814183 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 282/589] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:27.815193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07968f5d-d10e-450c-af17-cd45443b8813 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:27.825005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:27.838116 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:27.913716 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be83adb-ad5f-4b9d-a089-b36471a40f27 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:27.938155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07968f5d-d10e-450c-af17-cd45443b8813 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:27.953560 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c290fb-e248-476b-998b-284b573cef28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:27.954209 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c290fb-e248-476b-998b-284b573cef28 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:27.954492 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c290fb-e248-476b-998b-284b573cef28 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:27.954928 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c290fb-e248-476b-998b-284b573cef28 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:27.956009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c290fb-e248-476b-998b-284b573cef28 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:27.956325 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c290fb-e248-476b-998b-284b573cef28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:27.957797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c290fb-e248-476b-998b-284b573cef28 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:27.974250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28c3de-f5a0-4639-adf2-50b11a31625b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:27.974986 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28c3de-f5a0-4639-adf2-50b11a31625b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:27.975343 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28c3de-f5a0-4639-adf2-50b11a31625b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:27.975713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f28c3de-f5a0-4639-adf2-50b11a31625b None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:27.976571 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28c3de-f5a0-4639-adf2-50b11a31625b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:27.976958 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28c3de-f5a0-4639-adf2-50b11a31625b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:27.978647 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f28c3de-f5a0-4639-adf2-50b11a31625b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:27.981623 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 308/590] 10.222.0.39 () {56 vars in 1137 bytes} [Thu Jul 28 22:15:27 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:27.982690 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c290fb-e248-476b-998b-284b573cef28 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:27.997914 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 283/591] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:27.998856 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28c3de-f5a0-4639-adf2-50b11a31625b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:28.085999 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:28.095967 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:28.105882 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c290fb-e248-476b-998b-284b573cef28 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:28.132739 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f28c3de-f5a0-4639-adf2-50b11a31625b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:28.156064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84acaf3-2b93-4ac5-b2fb-014626494256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:28.156918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84acaf3-2b93-4ac5-b2fb-014626494256 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:28.157347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84acaf3-2b93-4ac5-b2fb-014626494256 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:28.157822 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84acaf3-2b93-4ac5-b2fb-014626494256 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:28.158669 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84acaf3-2b93-4ac5-b2fb-014626494256 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:28.159142 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84acaf3-2b93-4ac5-b2fb-014626494256 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:28.160727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84acaf3-2b93-4ac5-b2fb-014626494256 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:28.173473 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 309/592] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:28.174657 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84acaf3-2b93-4ac5-b2fb-014626494256 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:28.179802 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74610e-cd3c-45ce-ba3d-9c8e8a0b843c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:28.180456 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74610e-cd3c-45ce-ba3d-9c8e8a0b843c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:28.180736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74610e-cd3c-45ce-ba3d-9c8e8a0b843c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:28.181045 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74610e-cd3c-45ce-ba3d-9c8e8a0b843c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:28.196459 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74610e-cd3c-45ce-ba3d-9c8e8a0b843c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:28.199165 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d74610e-cd3c-45ce-ba3d-9c8e8a0b843c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:28.216883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-9d74610e-cd3c-45ce-ba3d-9c8e8a0b843c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:28.216883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:28.216883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:28.216883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:28.216883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:28.216883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:28.216883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:28.216883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:28.216883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:28.216883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:28.216883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:28.216883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:28.216883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:28.216883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:28.216883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:28.217997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:28.217997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:28.217997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:28.217997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:28.217997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:28.217997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:28.217997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:28.217997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:28.217997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:28.217997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:28.217997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:28.217997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:28.217997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:28.217997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:28.217997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:28.217997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:28.217997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:28.219254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:28.219254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:28.219254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:28.219254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:28.219254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 284/593] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:28.219254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74610e-cd3c-45ce-ba3d-9c8e8a0b843c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:28.314153 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84acaf3-2b93-4ac5-b2fb-014626494256 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:28.334283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d74610e-cd3c-45ce-ba3d-9c8e8a0b843c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:28.346901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:28.355155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ae969-5976-4ba3-8b9d-89ef22560c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:28.356053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ae969-5976-4ba3-8b9d-89ef22560c79 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:28.356500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ae969-5976-4ba3-8b9d-89ef22560c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:28.356922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ae969-5976-4ba3-8b9d-89ef22560c79 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:28.363137 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:28.365381 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ae969-5976-4ba3-8b9d-89ef22560c79 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:28.367130 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ae969-5976-4ba3-8b9d-89ef22560c79 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:28.374045 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe22b02-e286-4992-8526-dcb3d071577f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:28.374704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe22b02-e286-4992-8526-dcb3d071577f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:28.375042 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe22b02-e286-4992-8526-dcb3d071577f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:28.375325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe22b02-e286-4992-8526-dcb3d071577f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:28.382638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-5d7ae969-5976-4ba3-8b9d-89ef22560c79 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 22:15:28.382638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:28.382638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:28.382638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:28.382638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:28.382638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:28.382638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:28.382638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:28.382638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:28.382638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:28.382638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:28.382638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:28.382638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:28.382638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:28.382638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:28.382638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:28.383782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:28.383782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:28.383782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:28.383782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:28.383782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:28.383782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:28.383782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:28.383782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:28.383782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:28.383782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:28.383782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:28.383782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:28.383782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:28.383782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:28.383782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:28.383782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:28.383782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:28.383782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:28.385452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:28.385452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:28.385452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:28.385452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:28.385452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:28.385452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:28.385452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:28.385452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:28.385452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 22:15:28.385452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:28.386077 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe22b02-e286-4992-8526-dcb3d071577f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:28.386077 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe22b02-e286-4992-8526-dcb3d071577f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:28.387964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 310/594] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:28.388886 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ae969-5976-4ba3-8b9d-89ef22560c79 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:28.397659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-cbe22b02-e286-4992-8526-dcb3d071577f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:28.397659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:28.397659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:28.397659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:28.397659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:28.397659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:28.397659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:28.397659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:28.397659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:28.397659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:28.397659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:28.397659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:28.397659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:28.397659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:28.397659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:28.398767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:28.398767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:28.398767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:28.398767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:28.398767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:28.398767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:28.398767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:28.398767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:28.398767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:28.398767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:28.398767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:28.398767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:28.398767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:28.398767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:28.398767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:28.398767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:28.398767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:28.400716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:28.400716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:28.400716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:28.400716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:28.400716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:28.400716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:28.400716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:28.400716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:28.404229 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 285/595] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:28.405196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe22b02-e286-4992-8526-dcb3d071577f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:28.507584 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ae969-5976-4ba3-8b9d-89ef22560c79 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:28.511479 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe22b02-e286-4992-8526-dcb3d071577f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:28.547204 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539915f-2dbc-43ed-ac2a-ce26b1bbb3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:28.547420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b90a48-d968-4192-a7b2-ad48edafc9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:28.547980 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b90a48-d968-4192-a7b2-ad48edafc9ac None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:28.548270 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b90a48-d968-4192-a7b2-ad48edafc9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:28.548551 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b90a48-d968-4192-a7b2-ad48edafc9ac None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:28.548677 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539915f-2dbc-43ed-ac2a-ce26b1bbb3df None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:28.549094 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539915f-2dbc-43ed-ac2a-ce26b1bbb3df None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:28.549353 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b90a48-d968-4192-a7b2-ad48edafc9ac None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:28.549690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b90a48-d968-4192-a7b2-ad48edafc9ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:28.549828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539915f-2dbc-43ed-ac2a-ce26b1bbb3df None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:28.550648 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539915f-2dbc-43ed-ac2a-ce26b1bbb3df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:28.551101 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539915f-2dbc-43ed-ac2a-ce26b1bbb3df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:28.551208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b90a48-d968-4192-a7b2-ad48edafc9ac None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:28.552784 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539915f-2dbc-43ed-ac2a-ce26b1bbb3df None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:28.566695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 311/596] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:28.567773 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539915f-2dbc-43ed-ac2a-ce26b1bbb3df None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:28.570080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 286/597] 10.222.0.39 () {56 vars in 1139 bytes} [Thu Jul 28 22:15:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:28.577173 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b90a48-d968-4192-a7b2-ad48edafc9ac None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:28.624896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:28.631031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:28.679965 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539915f-2dbc-43ed-ac2a-ce26b1bbb3df None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:28.688554 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b90a48-d968-4192-a7b2-ad48edafc9ac None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:28.717909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56041b-2046-4d68-82f8-d2ca3768a508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:28.718802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56041b-2046-4d68-82f8-d2ca3768a508 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:28.719232 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56041b-2046-4d68-82f8-d2ca3768a508 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:28.719743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56041b-2046-4d68-82f8-d2ca3768a508 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:28.720783 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56041b-2046-4d68-82f8-d2ca3768a508 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:28.721232 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56041b-2046-4d68-82f8-d2ca3768a508 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:28.722834 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56041b-2046-4d68-82f8-d2ca3768a508 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:28.725704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463a5a7-a5e4-4685-84cb-b712738715eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:28.726191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463a5a7-a5e4-4685-84cb-b712738715eb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:28.726543 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463a5a7-a5e4-4685-84cb-b712738715eb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:28.726888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6463a5a7-a5e4-4685-84cb-b712738715eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:28.741319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463a5a7-a5e4-4685-84cb-b712738715eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:28.741585 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 312/598] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:28.742548 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56041b-2046-4d68-82f8-d2ca3768a508 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:28.750285 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6463a5a7-a5e4-4685-84cb-b712738715eb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:28.770403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-6463a5a7-a5e4-4685-84cb-b712738715eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:28.770403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:28.770403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:28.770403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:28.770403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:28.770403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:28.770403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:28.770403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:28.770403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:28.770403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:28.770403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:28.770403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:28.770403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:28.770403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:28.770403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:28.771657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:28.771657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:28.771657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:28.771657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:28.771657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:28.771657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:28.771657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:28.771657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:28.771657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:28.771657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:28.771657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:28.771657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:28.771657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:28.771657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:28.771657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:28.771657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:28.771657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:28.772681 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:28.772681 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:28.772681 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:28.772681 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:28.772681 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 287/599] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:28.772681 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463a5a7-a5e4-4685-84cb-b712738715eb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:28.862259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56041b-2046-4d68-82f8-d2ca3768a508 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:28.875627 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:28.889612 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:28.897743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6463a5a7-a5e4-4685-84cb-b712738715eb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:28.899855 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8350cb2-d604-4db9-b654-464b74e367ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:28.901285 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8350cb2-d604-4db9-b654-464b74e367ad None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:28.901285 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8350cb2-d604-4db9-b654-464b74e367ad None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:28.901285 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8350cb2-d604-4db9-b654-464b74e367ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:28.908496 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8350cb2-d604-4db9-b654-464b74e367ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:28.910108 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8350cb2-d604-4db9-b654-464b74e367ad None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:28.922724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-b8350cb2-d604-4db9-b654-464b74e367ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:28.922724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:28.922724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:28.922724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:28.922724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:28.922724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:28.922724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:28.922724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:28.922724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:28.922724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:28.922724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:28.922724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:28.922724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:28.922724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:28.922724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:28.923868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:28.923868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:28.923868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:28.923868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:28.923868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:28.923868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:28.923868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:28.923868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:28.923868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:28.923868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:28.923868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:28.923868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:28.923868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:28.923868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:28.923868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:28.923868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:28.923868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:28.924808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:28.924808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:28.924808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:28.924808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:28.924808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:28.924808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:28.924808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:28.924808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:28.924808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 313/600] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:28.925304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8350cb2-d604-4db9-b654-464b74e367ad None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:28.940899 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b32c8d-9687-45ab-80ce-880c262828ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:28.941648 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b32c8d-9687-45ab-80ce-880c262828ce None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:28.941940 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b32c8d-9687-45ab-80ce-880c262828ce None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:28.942218 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b32c8d-9687-45ab-80ce-880c262828ce None admin] PATH_INFO: `/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/b4aa320b18ce43c3acfece4546fea0f1/roles/a66ab77a63174592a1e468d2464e8bc4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:28.963941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b32c8d-9687-45ab-80ce-880c262828ce None admin] RBAC: Authorizing `identity:create_grant(project_id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user_id=b4aa320b18ce43c3acfece4546fea0f1, role_id=a66ab77a63174592a1e468d2464e8bc4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:28.966142 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b32c8d-9687-45ab-80ce-880c262828ce None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:28.997559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 288/601] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:15:28 2022] PUT /identity/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/b4aa320b18ce43c3acfece4546fea0f1/roles/a66ab77a63174592a1e468d2464e8bc4 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:15:28.997559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b32c8d-9687-45ab-80ce-880c262828ce None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:29.046767 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8350cb2-d604-4db9-b654-464b74e367ad None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:29.086955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3f23c-70dd-48cb-b4f1-a0d78e189984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:29.088154 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3f23c-70dd-48cb-b4f1-a0d78e189984 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:29.088423 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3f23c-70dd-48cb-b4f1-a0d78e189984 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:29.089170 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3f23c-70dd-48cb-b4f1-a0d78e189984 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:29.090032 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3f23c-70dd-48cb-b4f1-a0d78e189984 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:29.090344 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3f23c-70dd-48cb-b4f1-a0d78e189984 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:29.091844 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3f23c-70dd-48cb-b4f1-a0d78e189984 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:29.112773 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 314/602] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:29.114336 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04845bdf-5891-4222-b1bc-45642b3009e1 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:29.114632 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04845bdf-5891-4222-b1bc-45642b3009e1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:29.114909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04845bdf-5891-4222-b1bc-45642b3009e1 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:29.116551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 315/603] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:29.117315 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04845bdf-5891-4222-b1bc-45642b3009e1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:29.146181 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b32c8d-9687-45ab-80ce-880c262828ce None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:29.167627 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:29.188368 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da3143-89e3-4e72-a196-489c6ea15064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:29.188998 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da3143-89e3-4e72-a196-489c6ea15064 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:29.189281 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da3143-89e3-4e72-a196-489c6ea15064 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:29.189579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da3143-89e3-4e72-a196-489c6ea15064 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:29.190344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da3143-89e3-4e72-a196-489c6ea15064 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:29.190653 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da3143-89e3-4e72-a196-489c6ea15064 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:29.192085 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07da3143-89e3-4e72-a196-489c6ea15064 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:29.213406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 289/604] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:29.220180 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0603854-3286-40c8-99ef-4165c476ab5b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:29.220180 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0603854-3286-40c8-99ef-4165c476ab5b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:29.220180 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0603854-3286-40c8-99ef-4165c476ab5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:29.239563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:29.247671 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e0603854-3286-40c8-99ef-4165c476ab5b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:29.250219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e0603854-3286-40c8-99ef-4165c476ab5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:29.256935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04845bdf-5891-4222-b1bc-45642b3009e1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:29.298715 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d9059-7d8f-4a7e-84a2-17dc7d84d92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:29.299413 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d9059-7d8f-4a7e-84a2-17dc7d84d92f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:29.299701 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d9059-7d8f-4a7e-84a2-17dc7d84d92f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:29.299967 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d9059-7d8f-4a7e-84a2-17dc7d84d92f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:29.307933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d9059-7d8f-4a7e-84a2-17dc7d84d92f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:29.309496 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d9059-7d8f-4a7e-84a2-17dc7d84d92f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:29.322065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-e09d9059-7d8f-4a7e-84a2-17dc7d84d92f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:29.322065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:29.322065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:29.322065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:29.322065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:29.322065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:29.322065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:29.322065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:29.322065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:29.322065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:29.322065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:29.322065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:29.322065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:29.322065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:29.322065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:29.323000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:29.323000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:29.323000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:29.323000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:29.323000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:29.323000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:29.323000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:29.323000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:29.323000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:29.323000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:29.323000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:29.323000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:29.323000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:29.323000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:29.323000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:29.323000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:29.323000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:29.323920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:29.323920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:29.323920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:29.323920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:29.323920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:29.323920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:29.323920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:29.323920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:29.323920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 316/605] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:29.328967 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d9059-7d8f-4a7e-84a2-17dc7d84d92f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:29.343697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 290/606] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:29.349218 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0603854-3286-40c8-99ef-4165c476ab5b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:29.471518 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:29.472010 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0603854-3286-40c8-99ef-4165c476ab5b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:29.485494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d9059-7d8f-4a7e-84a2-17dc7d84d92f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:29.512214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241e7b7-8570-412b-8429-a28ae0751e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:29.512937 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241e7b7-8570-412b-8429-a28ae0751e70 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:29.513120 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241e7b7-8570-412b-8429-a28ae0751e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:29.513400 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1241e7b7-8570-412b-8429-a28ae0751e70 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:29.514490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241e7b7-8570-412b-8429-a28ae0751e70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:29.514746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241e7b7-8570-412b-8429-a28ae0751e70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:29.516246 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1241e7b7-8570-412b-8429-a28ae0751e70 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:29.518845 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:29.527545 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f9af9-bcd6-4097-a495-4c8fe9669092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:29.528584 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f9af9-bcd6-4097-a495-4c8fe9669092 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:29.529070 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f9af9-bcd6-4097-a495-4c8fe9669092 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:29.529567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f9af9-bcd6-4097-a495-4c8fe9669092 None admin] PATH_INFO: `/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/0fecb440c0874ede9f3cf544cfed93ab/roles/a66ab77a63174592a1e468d2464e8bc4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:29.536300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 291/607] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:29.537392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241e7b7-8570-412b-8429-a28ae0751e70 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:29.546708 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f9af9-bcd6-4097-a495-4c8fe9669092 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user_id=0fecb440c0874ede9f3cf544cfed93ab, role_id=a66ab77a63174592a1e468d2464e8bc4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:29.548524 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f9af9-bcd6-4097-a495-4c8fe9669092 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:29.565875 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 317/608] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:15:29 2022] PUT /identity/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/0fecb440c0874ede9f3cf544cfed93ab/roles/a66ab77a63174592a1e468d2464e8bc4 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:15:29.574750 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f9af9-bcd6-4097-a495-4c8fe9669092 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:29.653606 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1241e7b7-8570-412b-8429-a28ae0751e70 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:29.689059 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628eaa8-7c33-48b7-990d-80d9b3ae58b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:29.689751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628eaa8-7c33-48b7-990d-80d9b3ae58b5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:29.690039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628eaa8-7c33-48b7-990d-80d9b3ae58b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:29.690307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628eaa8-7c33-48b7-990d-80d9b3ae58b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:29.701949 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9628eaa8-7c33-48b7-990d-80d9b3ae58b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:29.707667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-9628eaa8-7c33-48b7-990d-80d9b3ae58b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:29.707667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:29.707667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:29.707667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:29.707667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:29.707667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:29.707667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:29.707667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:29.707667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:29.707667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:29.707667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:29.707667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:29.707667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:29.707667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:29.707667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:29.707667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:29.708637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:29.708637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:29.708637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:29.708637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:29.708637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:29.708637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:29.708637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:29.708637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:29.708637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:29.708637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:29.708637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:29.708637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:29.708637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:29.708637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:29.708637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:29.708637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:29.709576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:29.709576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:29.709576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:29.709576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:29.709576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:29.709576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:29.709576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:29.709576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:29.709576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:29.709576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:29.709576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:29.709576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:29.709576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 292/609] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:29.721678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628eaa8-7c33-48b7-990d-80d9b3ae58b5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:29.756544 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f9af9-bcd6-4097-a495-4c8fe9669092 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:29.780119 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:29.799246 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339b144-7e73-43e1-aefd-020df9989876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:29.800186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339b144-7e73-43e1-aefd-020df9989876 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:29.800614 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339b144-7e73-43e1-aefd-020df9989876 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:29.801063 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339b144-7e73-43e1-aefd-020df9989876 None admin] PATH_INFO: `/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/09b96431ac98447283960019cadde8a3/roles/a66ab77a63174592a1e468d2464e8bc4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:29.816147 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:29.819093 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339b144-7e73-43e1-aefd-020df9989876 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user_id=09b96431ac98447283960019cadde8a3, role_id=a66ab77a63174592a1e468d2464e8bc4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:29.820815 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339b144-7e73-43e1-aefd-020df9989876 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:29.835608 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9628eaa8-7c33-48b7-990d-80d9b3ae58b5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:29.839593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 318/610] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:15:29 2022] PUT /identity/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/09b96431ac98447283960019cadde8a3/roles/a66ab77a63174592a1e468d2464e8bc4 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:15:29.841193 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcfe34-9ccd-4b9c-9717-045bfb010a6e None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:29.841650 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcfe34-9ccd-4b9c-9717-045bfb010a6e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:29.842094 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dcfe34-9ccd-4b9c-9717-045bfb010a6e None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:29.848077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 319/611] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:29.849765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f10da5-087e-440d-82dc-0a76c27889d7 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:29.850328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f10da5-087e-440d-82dc-0a76c27889d7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:29.850732 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f10da5-087e-440d-82dc-0a76c27889d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:29.881894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-15f10da5-087e-440d-82dc-0a76c27889d7 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:29.883615 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-15f10da5-087e-440d-82dc-0a76c27889d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:29.899388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da04b13-3c04-41c4-9bf5-20d265dc17e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:29.900785 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da04b13-3c04-41c4-9bf5-20d265dc17e1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:29.901151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da04b13-3c04-41c4-9bf5-20d265dc17e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:29.901536 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da04b13-3c04-41c4-9bf5-20d265dc17e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:29.902870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da04b13-3c04-41c4-9bf5-20d265dc17e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:29.903279 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da04b13-3c04-41c4-9bf5-20d265dc17e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:29.904876 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da04b13-3c04-41c4-9bf5-20d265dc17e1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:29.915513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 293/612] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:15:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:29.922627 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da04b13-3c04-41c4-9bf5-20d265dc17e1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:29.975927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 320/613] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:29.999727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f10da5-087e-440d-82dc-0a76c27889d7 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:30.065200 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da04b13-3c04-41c4-9bf5-20d265dc17e1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:30.090196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:30.109707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5069f1d-1435-405d-862a-1f427513f604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:30.110665 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5069f1d-1435-405d-862a-1f427513f604 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:30.110665 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5069f1d-1435-405d-862a-1f427513f604 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:30.110962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5069f1d-1435-405d-862a-1f427513f604 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:30.122769 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5069f1d-1435-405d-862a-1f427513f604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:30.124237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5069f1d-1435-405d-862a-1f427513f604 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:30.140364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a5069f1d-1435-405d-862a-1f427513f604 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:30.140364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:30.140364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:30.140364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:30.140364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:30.140364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:30.140364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:30.140364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:30.140364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:30.140364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:30.140364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:30.140364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:30.140364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:30.140364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:30.140364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:30.141309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:30.141309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:30.141309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:30.141309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:30.141309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:30.141309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:30.141309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:30.141309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:30.141309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:30.141309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:30.141309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:30.141309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:30.141309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:30.141309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:30.141309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:30.141309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:30.141309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:30.142237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:30.142237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:30.142237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:30.142237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:30.142237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 294/614] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:30.147961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5069f1d-1435-405d-862a-1f427513f604 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:30.202611 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f10da5-087e-440d-82dc-0a76c27889d7 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:30.237916 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:30.267320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d9ec4-9ae0-480a-a13e-2d954f4409e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:30.268199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d9ec4-9ae0-480a-a13e-2d954f4409e8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:30.268583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d9ec4-9ae0-480a-a13e-2d954f4409e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:30.268958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d9ec4-9ae0-480a-a13e-2d954f4409e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:30.286144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d9ec4-9ae0-480a-a13e-2d954f4409e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:30.295161 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5069f1d-1435-405d-862a-1f427513f604 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:30.299036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-a38d9ec4-9ae0-480a-a13e-2d954f4409e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:30.299036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:30.299036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:30.299036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:30.299036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:30.299036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:30.299036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:30.299036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:30.299036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:30.299036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:30.299036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:30.299036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:30.299036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:30.299036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:30.299036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:30.299036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:30.300164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:30.300164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:30.300164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:30.300164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:30.300164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:30.300164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:30.300164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:30.300164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:30.300164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:30.300164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:30.300164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:30.300164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:30.300164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:30.300164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:30.300164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:30.300164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:30.301225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:30.301225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:30.301225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:30.301225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:30.301225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:30.301225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:30.301225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:30.301225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:30.301225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:30.301225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:30.301225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:30.301225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:30.309213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 321/615] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:15:30.313917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d9ec4-9ae0-480a-a13e-2d954f4409e8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:30.362892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8212b36-162d-4458-80d8-8bc6fa3f7201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:30.364965 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8212b36-162d-4458-80d8-8bc6fa3f7201 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:30.365363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8212b36-162d-4458-80d8-8bc6fa3f7201 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:30.365794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8212b36-162d-4458-80d8-8bc6fa3f7201 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:30.367469 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8212b36-162d-4458-80d8-8bc6fa3f7201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:30.368916 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8212b36-162d-4458-80d8-8bc6fa3f7201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:30.372154 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8212b36-162d-4458-80d8-8bc6fa3f7201 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:30.398287 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 295/616] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:30.403221 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8212b36-162d-4458-80d8-8bc6fa3f7201 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:30.466299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:30.511861 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d9ec4-9ae0-480a-a13e-2d954f4409e8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:30.588467 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8212b36-162d-4458-80d8-8bc6fa3f7201 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:30.591077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd331bcd-bd57-4c40-8ea7-4bd881acab97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:30.591699 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd331bcd-bd57-4c40-8ea7-4bd881acab97 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:30.591977 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd331bcd-bd57-4c40-8ea7-4bd881acab97 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:30.592251 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd331bcd-bd57-4c40-8ea7-4bd881acab97 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:30.592969 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd331bcd-bd57-4c40-8ea7-4bd881acab97 None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:30.593271 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd331bcd-bd57-4c40-8ea7-4bd881acab97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:30.594716 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd331bcd-bd57-4c40-8ea7-4bd881acab97 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:30.609333 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 322/617] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:15:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:30.614729 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd331bcd-bd57-4c40-8ea7-4bd881acab97 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:30.669151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832585f1-b6b7-46c9-9248-de72a2ca595a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:30.670082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832585f1-b6b7-46c9-9248-de72a2ca595a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:30.670592 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832585f1-b6b7-46c9-9248-de72a2ca595a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:30.671682 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832585f1-b6b7-46c9-9248-de72a2ca595a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:30.685668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832585f1-b6b7-46c9-9248-de72a2ca595a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:30.691736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832585f1-b6b7-46c9-9248-de72a2ca595a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:30.709872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:30.710800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-832585f1-b6b7-46c9-9248-de72a2ca595a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 22:15:30.710800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:30.710800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:30.710800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:30.710800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:30.710800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:30.710800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:30.710800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:30.710800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:30.710800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:30.710800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:30.710800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:30.710800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:30.710800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:30.710800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:30.710800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:30.711743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:30.711743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:30.711743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:30.711743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:30.711743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:30.711743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:30.711743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:30.711743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:30.711743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:30.711743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:30.711743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:30.711743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:30.711743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:30.711743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:30.711743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:30.711743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:30.711743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:30.711743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:30.713487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:30.713487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:30.713487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:30.713487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:30.713487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:30.713487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:30.713487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:30.713487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:30.713487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 22:15:30.713487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:30.713487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 296/618] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:15:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:30.718343 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832585f1-b6b7-46c9-9248-de72a2ca595a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:30.823793 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:30.835436 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832585f1-b6b7-46c9-9248-de72a2ca595a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:30.843415 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd331bcd-bd57-4c40-8ea7-4bd881acab97 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:30.874458 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3e150-b97e-4a48-83bf-4aa498be6f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:30.875085 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3e150-b97e-4a48-83bf-4aa498be6f9e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:30.875365 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3e150-b97e-4a48-83bf-4aa498be6f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:30.875635 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a3e150-b97e-4a48-83bf-4aa498be6f9e None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:30.876433 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a3e150-b97e-4a48-83bf-4aa498be6f9e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:30.876759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a3e150-b97e-4a48-83bf-4aa498be6f9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:30.881548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a3e150-b97e-4a48-83bf-4aa498be6f9e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:30.897622 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 297/619] 10.222.0.39 () {56 vars in 1135 bytes} [Thu Jul 28 22:15:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:30.905501 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3e150-b97e-4a48-83bf-4aa498be6f9e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:30.925600 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff641f-cc44-4b91-b7f0-a4622206c6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:30.926638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff641f-cc44-4b91-b7f0-a4622206c6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:30.927148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff641f-cc44-4b91-b7f0-a4622206c6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:30.927667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff641f-cc44-4b91-b7f0-a4622206c6b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:30.952140 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff641f-cc44-4b91-b7f0-a4622206c6b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:30.953979 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff641f-cc44-4b91-b7f0-a4622206c6b3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:30.983095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-3dff641f-cc44-4b91-b7f0-a4622206c6b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:30.983095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:30.983095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:30.983095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:30.983095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:30.983095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:30.983095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:30.983095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:30.983095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:30.983095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:30.983095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:30.983095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:30.983095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:30.983095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:30.983095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:30.984177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:30.984177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:30.984177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:30.984177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:30.984177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:30.984177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:30.984177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:30.984177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:30.984177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:30.984177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:30.984177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:30.984177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:30.984177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:30.984177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:30.984177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:30.984177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:30.984177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:30.985067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:30.985067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:30.985067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:30.985067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:30.994526 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 323/620] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:30.995817 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff641f-cc44-4b91-b7f0-a4622206c6b3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:31.017250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a3e150-b97e-4a48-83bf-4aa498be6f9e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:31.055280 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd53c65-8229-4e1e-bcb6-7332cee96131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:31.056089 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd53c65-8229-4e1e-bcb6-7332cee96131 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:31.056817 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd53c65-8229-4e1e-bcb6-7332cee96131 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:31.057145 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd53c65-8229-4e1e-bcb6-7332cee96131 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:31.067687 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd53c65-8229-4e1e-bcb6-7332cee96131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:31.069157 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd53c65-8229-4e1e-bcb6-7332cee96131 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:31.079284 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:31.086718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7dd53c65-8229-4e1e-bcb6-7332cee96131 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:31.086718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:31.086718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:31.086718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:31.086718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:31.086718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:31.086718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:31.086718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:31.086718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:31.086718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:31.086718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:31.086718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:31.086718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:31.086718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:31.086718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:31.088579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:31.088579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:31.088579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:31.088579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:31.088579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:31.088579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:31.088579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:31.088579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:31.088579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:31.088579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:31.088579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:31.088579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:31.088579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:31.088579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:31.088579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:31.088579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:31.088579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:31.089502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:31.089502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:31.089502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:31.089502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:31.091034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 298/621] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:31.095545 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd53c65-8229-4e1e-bcb6-7332cee96131 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:31.200110 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:31.211520 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff641f-cc44-4b91-b7f0-a4622206c6b3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:31.215088 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd53c65-8229-4e1e-bcb6-7332cee96131 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:31.248171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2ba09-72b9-4e0c-adf7-e92eea18bd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:31.248798 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2ba09-72b9-4e0c-adf7-e92eea18bd36 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:31.249082 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2ba09-72b9-4e0c-adf7-e92eea18bd36 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:31.249462 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2ba09-72b9-4e0c-adf7-e92eea18bd36 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:31.250392 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2ba09-72b9-4e0c-adf7-e92eea18bd36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:31.250756 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2ba09-72b9-4e0c-adf7-e92eea18bd36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:31.252191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220aaa0-d1d4-4a32-8d05-7947016912a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:31.252316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2ba09-72b9-4e0c-adf7-e92eea18bd36 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:31.252872 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220aaa0-d1d4-4a32-8d05-7947016912a8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:31.253160 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220aaa0-d1d4-4a32-8d05-7947016912a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:31.253435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220aaa0-d1d4-4a32-8d05-7947016912a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:31.254167 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220aaa0-d1d4-4a32-8d05-7947016912a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:31.254474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220aaa0-d1d4-4a32-8d05-7947016912a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:31.255834 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220aaa0-d1d4-4a32-8d05-7947016912a8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:31.265426 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 324/622] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:31.266832 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346848cb-f65a-4fc0-8ac5-e32319786b17 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:31.267127 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346848cb-f65a-4fc0-8ac5-e32319786b17 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:31.267410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346848cb-f65a-4fc0-8ac5-e32319786b17 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:31.269795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 299/624] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:31.272845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 325/624] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:31.275157 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e9853-106a-41d1-be3c-a7b044c9886b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:31.275485 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e9853-106a-41d1-be3c-a7b044c9886b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:31.276222 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220aaa0-d1d4-4a32-8d05-7947016912a8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:31.276888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177e9853-106a-41d1-be3c-a7b044c9886b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:31.295990 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-177e9853-106a-41d1-be3c-a7b044c9886b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:31.297209 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-177e9853-106a-41d1-be3c-a7b044c9886b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:31.350444 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:31.377651 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 326/625] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:31.378803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e9853-106a-41d1-be3c-a7b044c9886b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:31.399937 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220aaa0-d1d4-4a32-8d05-7947016912a8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:31.444947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874980d-2f0e-4e5e-917d-3d0772fcac4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:31.446112 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874980d-2f0e-4e5e-917d-3d0772fcac4b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:31.447315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874980d-2f0e-4e5e-917d-3d0772fcac4b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:31.447835 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5874980d-2f0e-4e5e-917d-3d0772fcac4b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:31.456524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874980d-2f0e-4e5e-917d-3d0772fcac4b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:31.458401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5874980d-2f0e-4e5e-917d-3d0772fcac4b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:31.473401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-5874980d-2f0e-4e5e-917d-3d0772fcac4b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 22:15:31.473401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:31.473401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:31.473401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:31.473401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:31.473401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:31.473401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:31.473401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:31.473401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:31.473401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:31.473401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:31.473401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:31.473401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:31.473401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:31.473401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:31.473401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:31.474476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:31.474476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:31.474476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:31.474476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:31.474476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:31.474476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:31.474476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:31.474476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:31.474476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:31.474476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:31.474476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:31.474476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:31.474476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:31.474476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:31.474476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:31.474476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:31.474476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:31.474476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:31.475437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:31.475437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:31.475437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:31.475437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:31.475437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:31.475437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:31.475437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:31.475437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:31.475437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 22:15:31.475437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:31.480058 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 300/626] 10.222.0.39 () {56 vars in 1093 bytes} [Thu Jul 28 22:15:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:31.481220 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874980d-2f0e-4e5e-917d-3d0772fcac4b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:31.602582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5874980d-2f0e-4e5e-917d-3d0772fcac4b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:31.629106 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:31.637977 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:31.650745 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd09ebf-edb8-4597-9e6a-2765e74d2745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:31.651851 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd09ebf-edb8-4597-9e6a-2765e74d2745 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:31.652151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd09ebf-edb8-4597-9e6a-2765e74d2745 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:31.652534 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd09ebf-edb8-4597-9e6a-2765e74d2745 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:31.657732 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177e9853-106a-41d1-be3c-a7b044c9886b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:31.665060 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd09ebf-edb8-4597-9e6a-2765e74d2745 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:31.672583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-3bd09ebf-edb8-4597-9e6a-2765e74d2745 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:31.672583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:31.672583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:31.672583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:31.672583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:31.672583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:31.672583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:31.672583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:31.672583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:31.672583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:31.672583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:31.672583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:31.672583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:31.672583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:31.672583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:31.672583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:31.673561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:31.673561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:31.673561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:31.673561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:31.673561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:31.673561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:31.673561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:31.673561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:31.673561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:31.673561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:31.673561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:31.673561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:31.673561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:31.673561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:31.673561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:31.673561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:31.674495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:31.674495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:31.674495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:31.674495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:31.674495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:31.674495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:31.674495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:31.674495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:31.674495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:31.674495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:31.674495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:31.674495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:31.677603 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 301/627] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:31.678697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd09ebf-edb8-4597-9e6a-2765e74d2745 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:31.704553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79574a-831f-40d1-afff-f650b01cbabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:31.705693 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79574a-831f-40d1-afff-f650b01cbabc None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:31.706289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79574a-831f-40d1-afff-f650b01cbabc None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:31.706793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79574a-831f-40d1-afff-f650b01cbabc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:31.716023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79574a-831f-40d1-afff-f650b01cbabc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:31.717937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79574a-831f-40d1-afff-f650b01cbabc None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:31.732145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-5e79574a-831f-40d1-afff-f650b01cbabc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:31.732145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:31.732145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:31.732145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:31.732145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:31.732145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:31.732145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:31.732145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:31.732145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:31.732145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:31.732145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:31.732145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:31.732145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:31.732145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:31.732145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:31.733171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:31.733171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:31.733171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:31.733171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:31.733171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:31.733171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:31.733171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:31.733171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:31.733171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:31.733171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:31.733171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:31.733171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:31.733171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:31.733171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:31.733171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:31.733171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:31.733171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:31.734109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:31.734109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:31.734109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:31.734109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:31.734109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:31.734109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:31.734109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:31.734109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:31.741910 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 327/628] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:31.743141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79574a-831f-40d1-afff-f650b01cbabc None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:31.814637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd09ebf-edb8-4597-9e6a-2765e74d2745 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:31.863633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d63ee2-b356-453d-a492-39bc20d9976d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:31.864908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d63ee2-b356-453d-a492-39bc20d9976d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:31.865629 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d63ee2-b356-453d-a492-39bc20d9976d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:31.866055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d63ee2-b356-453d-a492-39bc20d9976d None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:31.867194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d63ee2-b356-453d-a492-39bc20d9976d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:31.867652 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d63ee2-b356-453d-a492-39bc20d9976d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:31.873329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d63ee2-b356-453d-a492-39bc20d9976d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:31.890806 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79574a-831f-40d1-afff-f650b01cbabc None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:31.897125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 302/629] 10.222.0.39 () {56 vars in 1133 bytes} [Thu Jul 28 22:15:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:31.898253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d63ee2-b356-453d-a492-39bc20d9976d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:31.936993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6549e-76db-4b3c-92b1-f7ea1bb0c156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:31.938702 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6549e-76db-4b3c-92b1-f7ea1bb0c156 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:31.939099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6549e-76db-4b3c-92b1-f7ea1bb0c156 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:31.939487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f6549e-76db-4b3c-92b1-f7ea1bb0c156 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:31.940319 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6549e-76db-4b3c-92b1-f7ea1bb0c156 None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:31.940734 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6549e-76db-4b3c-92b1-f7ea1bb0c156 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:31.942235 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f6549e-76db-4b3c-92b1-f7ea1bb0c156 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:31.944817 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:31.955354 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 328/630] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:15:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:31.960097 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6549e-76db-4b3c-92b1-f7ea1bb0c156 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:31.973889 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:32.031836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d63ee2-b356-453d-a492-39bc20d9976d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:32.068084 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a9bcc4-ba53-49a0-9bc1-c99ab8dc2b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:32.068514 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a9bcc4-ba53-49a0-9bc1-c99ab8dc2b3b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:32.068793 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a9bcc4-ba53-49a0-9bc1-c99ab8dc2b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:32.069065 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a9bcc4-ba53-49a0-9bc1-c99ab8dc2b3b None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:32.069882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a9bcc4-ba53-49a0-9bc1-c99ab8dc2b3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:32.070146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a9bcc4-ba53-49a0-9bc1-c99ab8dc2b3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:32.072531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a9bcc4-ba53-49a0-9bc1-c99ab8dc2b3b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:32.092223 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 303/631] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:32.093040 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a9bcc4-ba53-49a0-9bc1-c99ab8dc2b3b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:32.098079 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f6549e-76db-4b3c-92b1-f7ea1bb0c156 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:32.145662 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7e3701-f921-4f07-bad5-258ccd351eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:32.146619 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e3701-f921-4f07-bad5-258ccd351eba None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:32.146903 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e3701-f921-4f07-bad5-258ccd351eba None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:32.147177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7e3701-f921-4f07-bad5-258ccd351eba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:32.215224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:32.220373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a9bcc4-ba53-49a0-9bc1-c99ab8dc2b3b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:32.221592 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e3701-f921-4f07-bad5-258ccd351eba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:32.223046 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7e3701-f921-4f07-bad5-258ccd351eba None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:32.233232 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:32.255214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc93ce-4903-4e9c-b8ce-2ae38e9b6412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:32.255845 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc93ce-4903-4e9c-b8ce-2ae38e9b6412 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:32.256120 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc93ce-4903-4e9c-b8ce-2ae38e9b6412 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:32.256391 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc93ce-4903-4e9c-b8ce-2ae38e9b6412 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:32.267311 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc93ce-4903-4e9c-b8ce-2ae38e9b6412 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:32.268727 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc93ce-4903-4e9c-b8ce-2ae38e9b6412 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:32.284361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-12fc93ce-4903-4e9c-b8ce-2ae38e9b6412 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:32.284361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:32.284361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:32.284361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:32.284361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:32.284361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:32.284361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:32.284361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:32.284361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:32.284361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:32.284361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:32.284361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:32.284361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:32.284361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:32.284361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:32.285253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:32.285253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:32.285253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:32.285253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:32.285253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:32.285253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:32.285253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:32.285253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:32.285253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:32.285253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:32.285253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:32.285253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:32.285253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:32.285253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:32.285253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:32.285253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:32.285253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:32.286168 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:32.286168 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:32.286168 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:32.286168 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:32.286168 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 304/632] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:32.286726 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29d6fc-c52a-4787-aa96-f80ecb096b8b None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:32.286992 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29d6fc-c52a-4787-aa96-f80ecb096b8b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:32.287267 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff29d6fc-c52a-4787-aa96-f80ecb096b8b None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:32.288754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 305/633] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:32.289428 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29d6fc-c52a-4787-aa96-f80ecb096b8b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:32.329909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 329/634] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:15:32.331170 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e1fac-c27e-4f02-a34c-cd4ed2780692 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:32.331457 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e1fac-c27e-4f02-a34c-cd4ed2780692 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:32.331730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e1fac-c27e-4f02-a34c-cd4ed2780692 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:32.332985 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 330/635] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:32.333710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e1fac-c27e-4f02-a34c-cd4ed2780692 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:32.394845 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff29d6fc-c52a-4787-aa96-f80ecb096b8b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:32.431001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8ec172-cb18-42c4-8025-723d29053214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:32.431590 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ec172-cb18-42c4-8025-723d29053214 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:32.431876 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ec172-cb18-42c4-8025-723d29053214 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:32.432143 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8ec172-cb18-42c4-8025-723d29053214 None admin] PATH_INFO: `/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/e3c9107872f147f6ab8698b67bb545af/roles/a66ab77a63174592a1e468d2464e8bc4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:32.436559 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e1fac-c27e-4f02-a34c-cd4ed2780692 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:32.445708 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:32.446995 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ec172-cb18-42c4-8025-723d29053214 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user_id=e3c9107872f147f6ab8698b67bb545af, role_id=a66ab77a63174592a1e468d2464e8bc4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:32.448532 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8ec172-cb18-42c4-8025-723d29053214 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:32.458461 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:32.464271 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 306/636] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:15:32 2022] PUT /identity/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/e3c9107872f147f6ab8698b67bb545af/roles/a66ab77a63174592a1e468d2464e8bc4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:15:32.466405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe8fa9-d07b-4e33-8359-e4d26120f397 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:32.466696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe8fa9-d07b-4e33-8359-e4d26120f397 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:32.466972 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abe8fa9-d07b-4e33-8359-e4d26120f397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:32.483074 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08518d6c-4589-40b4-b280-299d8297dd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:32.483837 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08518d6c-4589-40b4-b280-299d8297dd0d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:32.490462 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08518d6c-4589-40b4-b280-299d8297dd0d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:32.490462 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08518d6c-4589-40b4-b280-299d8297dd0d None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:32.490462 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08518d6c-4589-40b4-b280-299d8297dd0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:32.490462 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08518d6c-4589-40b4-b280-299d8297dd0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:32.490462 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08518d6c-4589-40b4-b280-299d8297dd0d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:32.491782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5abe8fa9-d07b-4e33-8359-e4d26120f397 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:32.491919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5abe8fa9-d07b-4e33-8359-e4d26120f397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:32.507814 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 331/637] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:32.507814 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08518d6c-4589-40b4-b280-299d8297dd0d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:32.585558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 307/638] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:32.585558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf2c0c-adcf-4bfa-b135-322cced04250 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:32.585558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf2c0c-adcf-4bfa-b135-322cced04250 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:32.585558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edf2c0c-adcf-4bfa-b135-322cced04250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:32.603959 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8edf2c0c-adcf-4bfa-b135-322cced04250 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:32.606810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8edf2c0c-adcf-4bfa-b135-322cced04250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:32.636900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08518d6c-4589-40b4-b280-299d8297dd0d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:32.671758 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff5e5c-1cec-461b-8352-0bb4a133a671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:32.672402 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff5e5c-1cec-461b-8352-0bb4a133a671 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:32.672689 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff5e5c-1cec-461b-8352-0bb4a133a671 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:32.672970 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff5e5c-1cec-461b-8352-0bb4a133a671 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:32.683165 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 308/639] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:32.683995 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf2c0c-adcf-4bfa-b135-322cced04250 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:32.691555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff5e5c-1cec-461b-8352-0bb4a133a671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:32.693009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbff5e5c-1cec-461b-8352-0bb4a133a671 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:32.716900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-fbff5e5c-1cec-461b-8352-0bb4a133a671 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:32.716900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:32.716900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:32.716900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:32.716900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:32.716900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:32.716900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:32.716900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:32.716900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:32.716900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:32.716900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:32.716900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:32.716900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:32.716900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:32.716900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:32.717944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:32.717944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:32.717944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:32.717944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:32.717944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:32.717944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:32.717944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:32.717944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:32.717944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:32.717944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:32.717944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:32.717944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:32.717944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:32.717944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:32.717944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:32.717944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:32.717944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:32.718835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:32.718835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:32.718835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:32.718835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:32.718835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 332/640] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:32.719858 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff5e5c-1cec-461b-8352-0bb4a133a671 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:32.748557 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:32.755352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:32.798493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edf2c0c-adcf-4bfa-b135-322cced04250 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:32.834699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01bdf69-b12c-47be-ae76-be482d0cfa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:32.835339 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bdf69-b12c-47be-ae76-be482d0cfa88 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:32.835617 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bdf69-b12c-47be-ae76-be482d0cfa88 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:32.835886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01bdf69-b12c-47be-ae76-be482d0cfa88 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:32.836212 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff5e5c-1cec-461b-8352-0bb4a133a671 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:32.843100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01bdf69-b12c-47be-ae76-be482d0cfa88 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:32.844548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01bdf69-b12c-47be-ae76-be482d0cfa88 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:32.856778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-b01bdf69-b12c-47be-ae76-be482d0cfa88 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 22:15:32.856778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:32.856778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:32.856778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:32.856778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:32.856778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:32.856778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:32.856778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:32.856778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:32.856778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:32.856778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:32.856778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:32.856778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:32.856778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:32.856778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:32.856778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:32.857813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:32.857813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:32.857813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:32.857813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:32.857813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:32.857813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:32.857813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:32.857813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:32.857813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:32.857813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:32.857813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:32.857813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:32.857813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:32.857813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:32.857813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:32.857813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:32.857813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:32.857813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:32.858751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:32.858751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:32.858751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:32.858751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:32.858751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:32.858751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:32.858751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:32.858751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:32.858751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 22:15:32.858751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:32.858751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 309/641] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:15:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:32.858751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01bdf69-b12c-47be-ae76-be482d0cfa88 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:32.875037 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33a7cf-a9b8-4ab6-88c6-78b3edc1810d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:32.875678 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33a7cf-a9b8-4ab6-88c6-78b3edc1810d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:32.876078 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33a7cf-a9b8-4ab6-88c6-78b3edc1810d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:32.876355 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a33a7cf-a9b8-4ab6-88c6-78b3edc1810d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:32.886994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a33a7cf-a9b8-4ab6-88c6-78b3edc1810d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:32.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-1a33a7cf-a9b8-4ab6-88c6-78b3edc1810d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:32.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:32.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:32.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:32.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:32.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:32.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:32.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:32.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:32.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:32.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:32.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:32.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:32.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:32.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:32.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:32.893589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:32.893589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:32.893589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:32.893589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:32.893589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:32.893589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:32.893589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:32.893589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:32.893589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:32.893589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:32.893589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:32.893589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:32.893589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:32.893589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:32.893589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:32.893589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:32.894586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:32.894586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:32.894586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:32.894586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:32.894586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:32.894586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:32.894586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:32.894586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:32.894586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:32.894586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:32.894586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:32.894586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:32.894586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 333/642] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:32.895393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33a7cf-a9b8-4ab6-88c6-78b3edc1810d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:32.978687 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01bdf69-b12c-47be-ae76-be482d0cfa88 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:32.981603 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:33.000473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:33.009556 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a33a7cf-a9b8-4ab6-88c6-78b3edc1810d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:33.017424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc77ac5-dd7d-42bd-8343-6c40dbf6a2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:33.018091 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc77ac5-dd7d-42bd-8343-6c40dbf6a2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:33.018403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc77ac5-dd7d-42bd-8343-6c40dbf6a2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:33.018675 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc77ac5-dd7d-42bd-8343-6c40dbf6a2f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:33.029427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc77ac5-dd7d-42bd-8343-6c40dbf6a2f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:33.034857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-bbc77ac5-dd7d-42bd-8343-6c40dbf6a2f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:33.034857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:33.034857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:33.034857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:33.034857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:33.034857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:33.034857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:33.034857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:33.034857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:33.034857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:33.034857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:33.034857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:33.034857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:33.034857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:33.034857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:33.034857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:33.035795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:33.035795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:33.035795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:33.035795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:33.035795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:33.035795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:33.035795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:33.035795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:33.035795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:33.035795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:33.035795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:33.035795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:33.035795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:33.035795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:33.035795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:33.035795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:33.036701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:33.036701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:33.036701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:33.036701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:33.036701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:33.036701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:33.036701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:33.036701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:33.036701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:33.036701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:33.036701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:33.036701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:33.036701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 310/643] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:33.037425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc77ac5-dd7d-42bd-8343-6c40dbf6a2f1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:33.057248 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b872f0-809a-4189-89fa-852858b5f847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:33.058250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b872f0-809a-4189-89fa-852858b5f847 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:33.058717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b872f0-809a-4189-89fa-852858b5f847 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:33.059178 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b872f0-809a-4189-89fa-852858b5f847 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:33.060296 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b872f0-809a-4189-89fa-852858b5f847 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:33.060806 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b872f0-809a-4189-89fa-852858b5f847 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:33.062998 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b872f0-809a-4189-89fa-852858b5f847 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:33.084084 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 334/644] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:33.085094 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b872f0-809a-4189-89fa-852858b5f847 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:33.169426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc77ac5-dd7d-42bd-8343-6c40dbf6a2f1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:33.194596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b872f0-809a-4189-89fa-852858b5f847 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:33.204223 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f6afe-5d92-49fe-8c2d-c7b971fc32a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:33.204847 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f6afe-5d92-49fe-8c2d-c7b971fc32a2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:33.205126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f6afe-5d92-49fe-8c2d-c7b971fc32a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:33.205397 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f6afe-5d92-49fe-8c2d-c7b971fc32a2 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:33.206241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f6afe-5d92-49fe-8c2d-c7b971fc32a2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:33.206547 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f6afe-5d92-49fe-8c2d-c7b971fc32a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:33.207976 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f6afe-5d92-49fe-8c2d-c7b971fc32a2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:33.226709 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 311/645] 10.222.0.39 () {56 vars in 1137 bytes} [Thu Jul 28 22:15:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:33.230688 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f6afe-5d92-49fe-8c2d-c7b971fc32a2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:33.232485 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce14995-b852-47cf-b06c-cf74d44ebd42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:33.233296 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce14995-b852-47cf-b06c-cf74d44ebd42 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:33.233681 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce14995-b852-47cf-b06c-cf74d44ebd42 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:33.233936 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce14995-b852-47cf-b06c-cf74d44ebd42 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:33.234657 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce14995-b852-47cf-b06c-cf74d44ebd42 None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:33.235605 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce14995-b852-47cf-b06c-cf74d44ebd42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:33.236317 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce14995-b852-47cf-b06c-cf74d44ebd42 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:33.244889 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 335/646] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:15:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:33.245519 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce14995-b852-47cf-b06c-cf74d44ebd42 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:33.248169 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:33.279976 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:33.347415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f6afe-5d92-49fe-8c2d-c7b971fc32a2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:33.351717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce14995-b852-47cf-b06c-cf74d44ebd42 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:33.388447 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6e69a-cdf0-4af1-96cf-4443876fdcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:33.388889 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b1052-f8d4-424f-84a9-8df7afa7eea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:33.389118 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6e69a-cdf0-4af1-96cf-4443876fdcc0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:33.389653 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6e69a-cdf0-4af1-96cf-4443876fdcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:33.389765 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b1052-f8d4-424f-84a9-8df7afa7eea4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:33.389935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6e69a-cdf0-4af1-96cf-4443876fdcc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:33.390030 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b1052-f8d4-424f-84a9-8df7afa7eea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:33.390309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b1052-f8d4-424f-84a9-8df7afa7eea4 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:33.391017 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b1052-f8d4-424f-84a9-8df7afa7eea4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:33.391321 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b1052-f8d4-424f-84a9-8df7afa7eea4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:33.393379 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b1052-f8d4-424f-84a9-8df7afa7eea4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:33.396724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a6e69a-cdf0-4af1-96cf-4443876fdcc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:33.398183 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a6e69a-cdf0-4af1-96cf-4443876fdcc0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:33.407245 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 312/647] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:15:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:33.407245 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b1052-f8d4-424f-84a9-8df7afa7eea4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:33.410638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-c5a6e69a-cdf0-4af1-96cf-4443876fdcc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:33.410638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:33.410638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:33.410638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:33.410638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:33.410638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:33.410638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:33.410638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:33.410638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:33.410638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:33.410638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:33.410638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:33.410638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:33.410638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:33.410638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:33.411572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:33.411572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:33.411572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:33.411572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:33.411572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:33.411572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:33.411572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:33.411572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:33.411572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:33.411572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:33.411572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:33.411572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:33.411572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:33.411572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:33.411572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:33.411572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:33.411572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:33.412515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:33.412515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:33.412515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:33.412515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:33.412515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:33.412515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:33.412515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:33.412515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:33.412515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 336/648] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:33.419289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6e69a-cdf0-4af1-96cf-4443876fdcc0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:33.483274 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:33.516248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:33.520415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b1052-f8d4-424f-84a9-8df7afa7eea4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:33.527906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6e69a-cdf0-4af1-96cf-4443876fdcc0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:33.558982 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbe6d2-0fc1-4984-8dfc-28a4e8aa22a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:33.559607 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbe6d2-0fc1-4984-8dfc-28a4e8aa22a1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:33.559889 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbe6d2-0fc1-4984-8dfc-28a4e8aa22a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:33.560164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbe6d2-0fc1-4984-8dfc-28a4e8aa22a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:33.566429 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9dd1a-6a32-47d1-a334-9c7563ce9506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:33.567064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9dd1a-6a32-47d1-a334-9c7563ce9506 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:33.567344 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9dd1a-6a32-47d1-a334-9c7563ce9506 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:33.567617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e9dd1a-6a32-47d1-a334-9c7563ce9506 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:33.580133 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9dd1a-6a32-47d1-a334-9c7563ce9506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:33.581633 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e9dd1a-6a32-47d1-a334-9c7563ce9506 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:33.597120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-94e9dd1a-6a32-47d1-a334-9c7563ce9506 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:33.597120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:33.597120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:33.597120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:33.597120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:33.597120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:33.597120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:33.597120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:33.597120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:33.597120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:33.597120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:33.597120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:33.597120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:33.597120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:33.597120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:33.598323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:33.598323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:33.598323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:33.598323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:33.598323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:33.598323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:33.598323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:33.598323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:33.598323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:33.598323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:33.598323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:33.598323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:33.598323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:33.598323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:33.598323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:33.598323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:33.598323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:33.599802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:33.599802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:33.599802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:33.599802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:33.599802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 337/649] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:33.599802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9dd1a-6a32-47d1-a334-9c7563ce9506 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:33.643866 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbe6d2-0fc1-4984-8dfc-28a4e8aa22a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:33.645320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bbe6d2-0fc1-4984-8dfc-28a4e8aa22a1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:33.710156 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e9dd1a-6a32-47d1-a334-9c7563ce9506 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:33.725410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:33.750228 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153e8ce-af86-4e22-b4c4-beb3573dd0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:33.750796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153e8ce-af86-4e22-b4c4-beb3573dd0a7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:33.751070 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153e8ce-af86-4e22-b4c4-beb3573dd0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:33.751352 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153e8ce-af86-4e22-b4c4-beb3573dd0a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:33.752357 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:33.760774 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 313/650] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:15:33.761561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbe6d2-0fc1-4984-8dfc-28a4e8aa22a1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:33.765271 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153e8ce-af86-4e22-b4c4-beb3573dd0a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:33.766770 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153e8ce-af86-4e22-b4c4-beb3573dd0a7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:33.782120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-8153e8ce-af86-4e22-b4c4-beb3573dd0a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:33.782120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:33.782120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:33.782120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:33.782120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:33.782120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:33.782120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:33.782120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:33.782120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:33.782120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:33.782120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:33.782120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:33.782120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:33.782120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:33.782120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:33.783017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:33.783017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:33.783017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:33.783017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:33.783017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:33.783017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:33.783017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:33.783017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:33.783017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:33.783017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:33.783017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:33.783017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:33.783017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:33.783017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:33.783017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:33.783017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:33.783017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:33.783906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:33.783906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:33.783906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:33.783906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:33.783906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 338/651] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:33.784192 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153e8ce-af86-4e22-b4c4-beb3573dd0a7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:33.876460 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbe6d2-0fc1-4984-8dfc-28a4e8aa22a1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:33.893008 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153e8ce-af86-4e22-b4c4-beb3573dd0a7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:33.915986 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36655c92-62ba-445e-9e89-de5e1132d15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:33.916605 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36655c92-62ba-445e-9e89-de5e1132d15d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:33.916884 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36655c92-62ba-445e-9e89-de5e1132d15d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:33.917338 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36655c92-62ba-445e-9e89-de5e1132d15d None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:33.917985 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36655c92-62ba-445e-9e89-de5e1132d15d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:33.919123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36655c92-62ba-445e-9e89-de5e1132d15d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:33.920552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36655c92-62ba-445e-9e89-de5e1132d15d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:33.928131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0cb9e-7fdf-4ddb-b9ba-7e030e8aa37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:33.928740 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0cb9e-7fdf-4ddb-b9ba-7e030e8aa37b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:33.929019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0cb9e-7fdf-4ddb-b9ba-7e030e8aa37b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:33.929290 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0cb9e-7fdf-4ddb-b9ba-7e030e8aa37b None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:33.930013 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0cb9e-7fdf-4ddb-b9ba-7e030e8aa37b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:33.930314 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0cb9e-7fdf-4ddb-b9ba-7e030e8aa37b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:33.932569 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0cb9e-7fdf-4ddb-b9ba-7e030e8aa37b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:33.935238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 314/652] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:33.936281 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36655c92-62ba-445e-9e89-de5e1132d15d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:33.949623 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 339/653] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:33.950499 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0cb9e-7fdf-4ddb-b9ba-7e030e8aa37b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:33.977102 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:34.014000 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:34.055506 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0cb9e-7fdf-4ddb-b9ba-7e030e8aa37b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:34.060911 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36655c92-62ba-445e-9e89-de5e1132d15d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:34.100539 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1967a-743c-491e-852d-845c63baeb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:34.101165 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1967a-743c-491e-852d-845c63baeb98 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:34.101449 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1967a-743c-491e-852d-845c63baeb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:34.101758 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1967a-743c-491e-852d-845c63baeb98 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:34.102480 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1967a-743c-491e-852d-845c63baeb98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:34.102601 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6adf3-f235-4856-be6d-b182eb58240d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:34.103246 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6adf3-f235-4856-be6d-b182eb58240d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:34.103349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1967a-743c-491e-852d-845c63baeb98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:34.103530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6adf3-f235-4856-be6d-b182eb58240d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:34.103805 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6adf3-f235-4856-be6d-b182eb58240d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:34.104645 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1967a-743c-491e-852d-845c63baeb98 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:34.115335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6adf3-f235-4856-be6d-b182eb58240d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:34.116855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6adf3-f235-4856-be6d-b182eb58240d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:34.118374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 340/654] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:34.119720 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1967a-743c-491e-852d-845c63baeb98 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:34.138526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-91c6adf3-f235-4856-be6d-b182eb58240d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:34.138526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:34.138526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:34.138526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:34.138526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:34.138526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:34.138526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:34.138526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:34.138526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:34.138526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:34.138526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:34.138526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:34.138526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:34.138526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:34.138526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:34.139736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:34.139736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:34.139736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:34.139736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:34.139736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:34.139736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:34.139736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:34.139736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:34.139736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:34.139736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:34.139736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:34.139736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:34.139736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:34.139736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:34.139736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:34.139736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:34.139736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:34.140632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:34.140632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:34.140632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:34.140632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:34.140632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 315/655] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:34.141586 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6adf3-f235-4856-be6d-b182eb58240d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:34.224726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:34.240586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1967a-743c-491e-852d-845c63baeb98 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:34.256459 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6adf3-f235-4856-be6d-b182eb58240d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:34.271019 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:34.278524 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284c657-75d3-4ac4-865b-d5fb6645cef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:34.279156 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284c657-75d3-4ac4-865b-d5fb6645cef6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:34.279440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284c657-75d3-4ac4-865b-d5fb6645cef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:34.279718 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c284c657-75d3-4ac4-865b-d5fb6645cef6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:34.280518 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c284c657-75d3-4ac4-865b-d5fb6645cef6 None admin] RBAC: Adding query filter params (role.id=a66ab77a63174592a1e468d2464e8bc4, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=77efe8046ea0468cb0c7c58c24fc4e75) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:34.280833 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c284c657-75d3-4ac4-865b-d5fb6645cef6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:34.282301 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c284c657-75d3-4ac4-865b-d5fb6645cef6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:34.288327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 341/656] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:15:34 2022] GET /identity/v3/role_assignments?user.id=77efe8046ea0468cb0c7c58c24fc4e75&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=a66ab77a63174592a1e468d2464e8bc4 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:34.289195 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284c657-75d3-4ac4-865b-d5fb6645cef6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:34.310971 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48418bd-96f2-491b-bf41-21bbbece951b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:34.311684 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48418bd-96f2-491b-bf41-21bbbece951b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:34.311966 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48418bd-96f2-491b-bf41-21bbbece951b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:34.312233 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48418bd-96f2-491b-bf41-21bbbece951b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:34.322520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48418bd-96f2-491b-bf41-21bbbece951b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:34.323655 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48418bd-96f2-491b-bf41-21bbbece951b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:34.338300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a48418bd-96f2-491b-bf41-21bbbece951b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 22:15:34.338300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:34.338300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:34.338300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:34.338300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:34.338300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:34.338300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:34.338300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:34.338300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:34.338300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:34.338300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:34.338300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:34.338300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:34.338300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:34.338300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:34.338300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:34.339582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:34.339582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:34.339582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:34.339582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:34.339582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:34.339582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:34.339582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:34.339582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:34.339582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:34.339582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:34.339582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:34.339582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:34.339582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:34.339582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:34.339582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:34.339582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:34.339582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:34.339582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:34.340777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:34.340777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:34.340777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:34.340777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:34.340777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:34.340777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:34.340777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:34.340777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:34.340777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 28 22:15:34.340777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:34.340777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 316/657] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:34.340777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48418bd-96f2-491b-bf41-21bbbece951b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:34.451158 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c284c657-75d3-4ac4-865b-d5fb6645cef6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:34.463506 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48418bd-96f2-491b-bf41-21bbbece951b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:34.507893 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed504ac7-38bf-4c90-9967-cd187327d58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:34.510865 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed504ac7-38bf-4c90-9967-cd187327d58f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:34.511171 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4d6e8-e3f9-4ebf-a440-dfb8fd7ecd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:34.511505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed504ac7-38bf-4c90-9967-cd187327d58f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:34.511912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4d6e8-e3f9-4ebf-a440-dfb8fd7ecd3b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:34.512080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed504ac7-38bf-4c90-9967-cd187327d58f None admin] PATH_INFO: `/v3/users/glance` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:34.512195 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4d6e8-e3f9-4ebf-a440-dfb8fd7ecd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:34.512464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4d6e8-e3f9-4ebf-a440-dfb8fd7ecd3b None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:34.513173 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4d6e8-e3f9-4ebf-a440-dfb8fd7ecd3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:34.513483 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4d6e8-e3f9-4ebf-a440-dfb8fd7ecd3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:34.515209 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4d6e8-e3f9-4ebf-a440-dfb8fd7ecd3b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:34.521985 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed504ac7-38bf-4c90-9967-cd187327d58f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:34.523616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed504ac7-38bf-4c90-9967-cd187327d58f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:34.528899 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 317/658] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:34.530052 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4d6e8-e3f9-4ebf-a440-dfb8fd7ecd3b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:34.536058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ed504ac7-38bf-4c90-9967-cd187327d58f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 28 22:15:34.536058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:34.536058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:34.536058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:34.536058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:34.536058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:34.536058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:34.536058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:34.536058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:34.536058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:34.536058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:34.536058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:34.536058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:34.536058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:34.536058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:34.536058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:34.537019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:34.537019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:34.537019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:34.537019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:34.537019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:34.537019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:34.537019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:34.537019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:34.537019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:34.537019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:34.537019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:34.537019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:34.537019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:34.537019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:34.537019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:34.537019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:34.537019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:34.537019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:34.538017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:34.538017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:34.538017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:34.538017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:34.538017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:34.538017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:34.538017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:34.538017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:34.538017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 28 22:15:34.538017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:34.538017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 342/659] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:15:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:34.538630 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afce6c8-3341-40d1-b829-d951b40bdf01 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:34.538707 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afce6c8-3341-40d1-b829-d951b40bdf01 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:34.539329 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afce6c8-3341-40d1-b829-d951b40bdf01 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:34.542298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 343/660] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:34.542298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afce6c8-3341-40d1-b829-d951b40bdf01 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:34.555346 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:34.567750 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:34.645065 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4d6e8-e3f9-4ebf-a440-dfb8fd7ecd3b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:34.661246 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afce6c8-3341-40d1-b829-d951b40bdf01 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:34.682615 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31fb82-27fe-457f-99c5-49f988d97bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:34.683513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31fb82-27fe-457f-99c5-49f988d97bed None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:34.683798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31fb82-27fe-457f-99c5-49f988d97bed None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:34.684072 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e31fb82-27fe-457f-99c5-49f988d97bed None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:34.685344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31fb82-27fe-457f-99c5-49f988d97bed None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:34.685686 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31fb82-27fe-457f-99c5-49f988d97bed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:34.687552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e31fb82-27fe-457f-99c5-49f988d97bed None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:34.695324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc1e28-0831-4cf9-ba49-5d0633f31339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:34.696008 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc1e28-0831-4cf9-ba49-5d0633f31339 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:34.696369 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc1e28-0831-4cf9-ba49-5d0633f31339 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:34.697188 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc1e28-0831-4cf9-ba49-5d0633f31339 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:34.704855 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc1e28-0831-4cf9-ba49-5d0633f31339 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:34.705882 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc1e28-0831-4cf9-ba49-5d0633f31339 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:34.713682 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 318/661] 10.222.0.39 () {56 vars in 1139 bytes} [Thu Jul 28 22:15:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:34.713682 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31fb82-27fe-457f-99c5-49f988d97bed None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:34.717901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-b0dc1e28-0831-4cf9-ba49-5d0633f31339 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:34.717901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:34.717901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:34.717901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:34.717901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:34.717901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:34.717901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:34.717901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:34.717901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:34.717901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:34.717901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:34.717901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:34.717901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:34.717901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:34.717901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:34.718795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:34.718795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:34.718795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:34.718795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:34.718795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:34.718795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:34.718795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:34.718795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:34.718795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:34.718795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:34.718795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:34.718795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:34.718795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:34.718795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:34.718795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:34.718795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:34.718795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:34.719752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:34.719752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:34.719752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:34.719752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:34.719752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:34.719752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:34.719752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:34.719752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:34.719752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 344/662] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:34.720764 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89412154-6ea3-408d-bc8d-e47350b080d9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:34.720944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89412154-6ea3-408d-bc8d-e47350b080d9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:34.721168 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89412154-6ea3-408d-bc8d-e47350b080d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:34.739702 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-89412154-6ea3-408d-bc8d-e47350b080d9 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:34.740907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-89412154-6ea3-408d-bc8d-e47350b080d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:34.810204 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:34.811175 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:34.821708 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 345/663] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:34.821708 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89412154-6ea3-408d-bc8d-e47350b080d9 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:34.824289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e31fb82-27fe-457f-99c5-49f988d97bed None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:34.860531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae522f8-5c63-4aca-8f2c-e89b2aac6e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:34.861143 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae522f8-5c63-4aca-8f2c-e89b2aac6e94 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:34.861559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae522f8-5c63-4aca-8f2c-e89b2aac6e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:34.861839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae522f8-5c63-4aca-8f2c-e89b2aac6e94 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:34.862635 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae522f8-5c63-4aca-8f2c-e89b2aac6e94 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:34.862937 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae522f8-5c63-4aca-8f2c-e89b2aac6e94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:34.864360 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae522f8-5c63-4aca-8f2c-e89b2aac6e94 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:34.882295 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 319/664] 10.222.0.39 () {56 vars in 1137 bytes} [Thu Jul 28 22:15:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:34.883125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae522f8-5c63-4aca-8f2c-e89b2aac6e94 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:34.942069 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89412154-6ea3-408d-bc8d-e47350b080d9 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:34.980186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34729bff-b736-4850-a295-5557ad594290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:34.980944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34729bff-b736-4850-a295-5557ad594290 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:34.981286 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34729bff-b736-4850-a295-5557ad594290 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:34.981650 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34729bff-b736-4850-a295-5557ad594290 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:34.988293 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae522f8-5c63-4aca-8f2c-e89b2aac6e94 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:35.024179 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c0e3f-7b10-4831-90ef-311a12461413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:35.025564 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c0e3f-7b10-4831-90ef-311a12461413 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:35.025849 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c0e3f-7b10-4831-90ef-311a12461413 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:35.026114 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c0e3f-7b10-4831-90ef-311a12461413 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:35.026865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c0e3f-7b10-4831-90ef-311a12461413 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:35.027166 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c0e3f-7b10-4831-90ef-311a12461413 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:35.028595 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c0e3f-7b10-4831-90ef-311a12461413 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:35.040591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 320/665] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:35.041400 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c0e3f-7b10-4831-90ef-311a12461413 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:35.055960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:35.059803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:35.063036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34729bff-b736-4850-a295-5557ad594290 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:35.064964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34729bff-b736-4850-a295-5557ad594290 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:35.157697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c0e3f-7b10-4831-90ef-311a12461413 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:35.186210 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 346/666] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:15:35.187514 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34729bff-b736-4850-a295-5557ad594290 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:35.198813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8d262-3ce4-4a1d-90ad-16858abb37ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:35.199760 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8d262-3ce4-4a1d-90ad-16858abb37ec None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:35.200060 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8d262-3ce4-4a1d-90ad-16858abb37ec None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:35.200341 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8d262-3ce4-4a1d-90ad-16858abb37ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:35.210905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df8d262-3ce4-4a1d-90ad-16858abb37ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:35.216335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-4df8d262-3ce4-4a1d-90ad-16858abb37ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:35.216335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:35.216335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:35.216335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:35.216335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:35.216335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:35.216335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:35.216335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:35.216335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:35.216335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:35.216335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:35.216335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:35.216335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:35.216335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:35.216335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:35.216335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:35.217481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:35.217481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:35.217481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:35.217481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:35.217481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:35.217481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:35.217481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:35.217481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:35.217481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:35.217481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:35.217481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:35.217481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:35.217481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:35.217481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:35.217481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:35.217481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:35.218577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:35.218577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:35.218577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:35.218577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:35.218577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:35.218577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:35.218577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:35.218577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:35.218577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:35.218577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:35.218577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:35.218577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:35.218577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 321/667] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:35.219436 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8d262-3ce4-4a1d-90ad-16858abb37ec None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:35.293399 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:35.298258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:35.301263 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34729bff-b736-4850-a295-5557ad594290 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:35.328838 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8d262-3ce4-4a1d-90ad-16858abb37ec None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:35.338280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f4b1d-b81f-4540-a736-66ed7f54e8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:35.338905 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f4b1d-b81f-4540-a736-66ed7f54e8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:35.339188 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f4b1d-b81f-4540-a736-66ed7f54e8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:35.339464 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87f4b1d-b81f-4540-a736-66ed7f54e8b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:35.364798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2da619-703c-4290-917a-310c1fb44000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:35.365406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2da619-703c-4290-917a-310c1fb44000 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:35.365721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2da619-703c-4290-917a-310c1fb44000 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:35.365990 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2da619-703c-4290-917a-310c1fb44000 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:35.366786 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2da619-703c-4290-917a-310c1fb44000 None admin] RBAC: Adding query filter params (role.id=a66ab77a63174592a1e468d2464e8bc4, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=b4aa320b18ce43c3acfece4546fea0f1) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:35.367089 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2da619-703c-4290-917a-310c1fb44000 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:35.369054 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2da619-703c-4290-917a-310c1fb44000 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:35.375075 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 322/668] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:15:35 2022] GET /identity/v3/role_assignments?user.id=b4aa320b18ce43c3acfece4546fea0f1&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=a66ab77a63174592a1e468d2464e8bc4 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:35.375750 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2da619-703c-4290-917a-310c1fb44000 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:35.438943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f4b1d-b81f-4540-a736-66ed7f54e8b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:35.440753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87f4b1d-b81f-4540-a736-66ed7f54e8b9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:35.497092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2da619-703c-4290-917a-310c1fb44000 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:35.539452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:35.548296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f5b85-5cef-41f2-a51e-7a8b70d7297d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:35.549238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f5b85-5cef-41f2-a51e-7a8b70d7297d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:35.550085 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f5b85-5cef-41f2-a51e-7a8b70d7297d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:35.550660 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5f5b85-5cef-41f2-a51e-7a8b70d7297d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:35.564594 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f5b85-5cef-41f2-a51e-7a8b70d7297d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:35.567271 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5f5b85-5cef-41f2-a51e-7a8b70d7297d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:35.577255 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:35.583287 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 347/669] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:15:35.584113 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f4b1d-b81f-4540-a736-66ed7f54e8b9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:35.586523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ab5f5b85-5cef-41f2-a51e-7a8b70d7297d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:35.586523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:35.586523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:35.586523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:35.586523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:35.586523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:35.586523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:35.586523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:35.586523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:35.586523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:35.586523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:35.586523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:35.586523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:35.586523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:35.586523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:35.588018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:35.588018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:35.588018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:35.588018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:35.588018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:35.588018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:35.588018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:35.588018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:35.588018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:35.588018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:35.588018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:35.588018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:35.588018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:35.588018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:35.588018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:35.588018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:35.588018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:35.588924 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:35.588924 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:35.588924 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:35.588924 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:35.588924 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 323/670] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:35.589696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f5b85-5cef-41f2-a51e-7a8b70d7297d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:35.700227 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87f4b1d-b81f-4540-a736-66ed7f54e8b9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:35.714287 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5f5b85-5cef-41f2-a51e-7a8b70d7297d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:35.737065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2110e-7bc2-410b-92e0-f0089cc318f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:35.738078 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2110e-7bc2-410b-92e0-f0089cc318f0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:35.738580 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2110e-7bc2-410b-92e0-f0089cc318f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:35.739163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2110e-7bc2-410b-92e0-f0089cc318f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:35.740182 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2110e-7bc2-410b-92e0-f0089cc318f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:35.740696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2110e-7bc2-410b-92e0-f0089cc318f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:35.742388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad2110e-7bc2-410b-92e0-f0089cc318f0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:35.751474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2107eba-0066-4cbb-b19f-d6c93ce1a01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:35.752109 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2107eba-0066-4cbb-b19f-d6c93ce1a01a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:35.752440 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2107eba-0066-4cbb-b19f-d6c93ce1a01a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:35.752718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2107eba-0066-4cbb-b19f-d6c93ce1a01a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:35.758466 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 348/671] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:15:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:35.759825 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2110e-7bc2-410b-92e0-f0089cc318f0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:35.766401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2107eba-0066-4cbb-b19f-d6c93ce1a01a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:35.767866 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2107eba-0066-4cbb-b19f-d6c93ce1a01a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:35.783814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a2107eba-0066-4cbb-b19f-d6c93ce1a01a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:35.783814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:35.783814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:35.783814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:35.783814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:35.783814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:35.783814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:35.783814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:35.783814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:35.783814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:35.783814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:35.783814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:35.783814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:35.783814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:35.783814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:35.784763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:35.784763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:35.784763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:35.784763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:35.784763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:35.784763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:35.784763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:35.784763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:35.784763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:35.784763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:35.784763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:35.784763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:35.784763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:35.784763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:35.784763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:35.784763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:35.784763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:35.785704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:35.785704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:35.785704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:35.785704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:35.789732 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 324/672] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:35.791636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2107eba-0066-4cbb-b19f-d6c93ce1a01a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:35.830142 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:35.831423 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:35.873056 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad2110e-7bc2-410b-92e0-f0089cc318f0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:35.903607 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2107eba-0066-4cbb-b19f-d6c93ce1a01a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:35.909419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727da202-2ce3-4e1b-9851-758dddc3bd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:35.910381 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727da202-2ce3-4e1b-9851-758dddc3bd7a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:35.910896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727da202-2ce3-4e1b-9851-758dddc3bd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:35.911467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727da202-2ce3-4e1b-9851-758dddc3bd7a None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:35.912368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727da202-2ce3-4e1b-9851-758dddc3bd7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:35.912885 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727da202-2ce3-4e1b-9851-758dddc3bd7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:35.914479 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727da202-2ce3-4e1b-9851-758dddc3bd7a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:35.927374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 349/673] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:35.928507 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727da202-2ce3-4e1b-9851-758dddc3bd7a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:35.944091 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5806c01-b2a5-4cde-a4dd-301e3db383ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:35.944740 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5806c01-b2a5-4cde-a4dd-301e3db383ea None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:35.945018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5806c01-b2a5-4cde-a4dd-301e3db383ea None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:35.945291 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5806c01-b2a5-4cde-a4dd-301e3db383ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:35.957861 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5806c01-b2a5-4cde-a4dd-301e3db383ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:35.959694 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5806c01-b2a5-4cde-a4dd-301e3db383ea None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:35.975455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-e5806c01-b2a5-4cde-a4dd-301e3db383ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:35.975455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:35.975455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:35.975455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:35.975455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:35.975455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:35.975455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:35.975455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:35.975455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:35.975455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:35.975455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:35.975455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:35.975455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:35.975455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:35.975455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:35.976406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:35.976406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:35.976406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:35.976406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:35.976406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:35.976406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:35.976406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:35.976406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:35.976406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:35.976406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:35.976406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:35.976406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:35.976406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:35.976406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:35.976406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:35.976406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:35.976406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:35.977310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:35.977310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:35.977310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:35.977310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:35.977310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 325/674] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:35.978662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5806c01-b2a5-4cde-a4dd-301e3db383ea None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:36.037381 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727da202-2ce3-4e1b-9851-758dddc3bd7a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:36.062785 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:36.075741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1aee43-308e-4641-b929-5c808e9dd208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:36.076692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1aee43-308e-4641-b929-5c808e9dd208 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:36.077204 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1aee43-308e-4641-b929-5c808e9dd208 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:36.077918 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:36.079238 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1aee43-308e-4641-b929-5c808e9dd208 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:36.080202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1aee43-308e-4641-b929-5c808e9dd208 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:36.080723 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1aee43-308e-4641-b929-5c808e9dd208 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:36.082481 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1aee43-308e-4641-b929-5c808e9dd208 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:36.088409 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5806c01-b2a5-4cde-a4dd-301e3db383ea None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:36.101403 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 350/675] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:36.103271 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1aee43-308e-4641-b929-5c808e9dd208 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:36.131896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73037d6c-3d83-49b8-b92f-0219b0a5555b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:36.132466 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73037d6c-3d83-49b8-b92f-0219b0a5555b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:36.132750 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73037d6c-3d83-49b8-b92f-0219b0a5555b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:36.133022 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73037d6c-3d83-49b8-b92f-0219b0a5555b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:36.143251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73037d6c-3d83-49b8-b92f-0219b0a5555b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:36.146876 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73037d6c-3d83-49b8-b92f-0219b0a5555b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:36.160810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-73037d6c-3d83-49b8-b92f-0219b0a5555b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:36.160810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:36.160810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:36.160810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:36.160810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:36.160810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:36.160810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:36.160810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:36.160810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:36.160810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:36.160810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:36.160810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:36.160810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:36.160810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:36.160810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:36.161798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:36.161798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:36.161798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:36.161798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:36.161798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:36.161798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:36.161798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:36.161798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:36.161798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:36.161798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:36.161798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:36.161798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:36.161798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:36.161798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:36.161798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:36.161798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:36.161798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:36.162742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:36.162742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:36.162742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:36.162742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:36.162742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:36.162742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:36.162742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:36.162742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:36.168077 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 326/676] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:36.168415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73037d6c-3d83-49b8-b92f-0219b0a5555b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:36.225013 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1aee43-308e-4641-b929-5c808e9dd208 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:36.263889 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070f934-9367-4da6-a7d3-6310608f55ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:36.265240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070f934-9367-4da6-a7d3-6310608f55ff None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:36.265776 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070f934-9367-4da6-a7d3-6310608f55ff None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:36.266268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070f934-9367-4da6-a7d3-6310608f55ff None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:36.267329 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070f934-9367-4da6-a7d3-6310608f55ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:36.267883 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070f934-9367-4da6-a7d3-6310608f55ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:36.269599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070f934-9367-4da6-a7d3-6310608f55ff None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:36.273214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73037d6c-3d83-49b8-b92f-0219b0a5555b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:36.284182 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 351/677] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:36.285297 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070f934-9367-4da6-a7d3-6310608f55ff None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:36.309074 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbe91f-3566-438a-88dc-5f082af48345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:36.311926 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbe91f-3566-438a-88dc-5f082af48345 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:36.311926 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbe91f-3566-438a-88dc-5f082af48345 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:36.311926 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcbe91f-3566-438a-88dc-5f082af48345 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:36.318687 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcbe91f-3566-438a-88dc-5f082af48345 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:36.320133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcbe91f-3566-438a-88dc-5f082af48345 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:36.332558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-6dcbe91f-3566-438a-88dc-5f082af48345 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:36.332558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:36.332558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:36.332558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:36.332558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:36.332558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:36.332558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:36.332558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:36.332558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:36.332558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:36.332558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:36.332558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:36.332558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:36.332558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:36.332558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:36.333515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:36.333515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:36.333515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:36.333515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:36.333515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:36.333515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:36.333515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:36.333515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:36.333515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:36.333515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:36.333515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:36.333515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:36.333515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:36.333515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:36.333515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:36.333515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:36.333515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:36.334454 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:36.334454 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:36.334454 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:36.334454 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:36.334454 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:36.334454 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:36.334454 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:36.334454 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:36.337837 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 327/678] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:36.338736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbe91f-3566-438a-88dc-5f082af48345 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:36.341630 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:36.351153 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:36.408863 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070f934-9367-4da6-a7d3-6310608f55ff None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:36.445705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcbe91f-3566-438a-88dc-5f082af48345 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:36.452892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f10e8-7e95-4b8a-9708-d5f6b9541bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:36.453891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f10e8-7e95-4b8a-9708-d5f6b9541bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:36.454410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f10e8-7e95-4b8a-9708-d5f6b9541bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:36.454883 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3f10e8-7e95-4b8a-9708-d5f6b9541bb0 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:36.455956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f10e8-7e95-4b8a-9708-d5f6b9541bb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:36.456464 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f10e8-7e95-4b8a-9708-d5f6b9541bb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:36.458204 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3f10e8-7e95-4b8a-9708-d5f6b9541bb0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:36.471896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 352/679] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:36.473108 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f10e8-7e95-4b8a-9708-d5f6b9541bb0 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:36.487389 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef72047-94bd-49bc-b06e-ff359639da1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:36.487856 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef72047-94bd-49bc-b06e-ff359639da1a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:36.488141 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef72047-94bd-49bc-b06e-ff359639da1a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:36.488417 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef72047-94bd-49bc-b06e-ff359639da1a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:36.497793 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef72047-94bd-49bc-b06e-ff359639da1a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:36.499353 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef72047-94bd-49bc-b06e-ff359639da1a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:36.511520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-aef72047-94bd-49bc-b06e-ff359639da1a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 22:15:36.511520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:36.511520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:36.511520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:36.511520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:36.511520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:36.511520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:36.511520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:36.511520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:36.511520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:36.511520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:36.511520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:36.511520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:36.511520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:36.511520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:36.511520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:36.512470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:36.512470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:36.512470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:36.512470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:36.512470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:36.512470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:36.512470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:36.512470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:36.512470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:36.512470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:36.512470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:36.512470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:36.512470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:36.512470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:36.512470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:36.512470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:36.512470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:36.512470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:36.513447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:36.513447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:36.513447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:36.513447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:36.513447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:36.513447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:36.513447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:36.513447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:36.513447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 22:15:36.513447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:36.513447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 328/680] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:15:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:36.513447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef72047-94bd-49bc-b06e-ff359639da1a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:36.585897 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:36.589089 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3f10e8-7e95-4b8a-9708-d5f6b9541bb0 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:36.591753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:36.623660 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef72047-94bd-49bc-b06e-ff359639da1a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:36.633757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529783a8-01d6-4762-b63d-89f29049c440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:36.634785 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529783a8-01d6-4762-b63d-89f29049c440 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:36.635066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529783a8-01d6-4762-b63d-89f29049c440 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:36.635606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529783a8-01d6-4762-b63d-89f29049c440 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:36.636367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529783a8-01d6-4762-b63d-89f29049c440 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:36.636676 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529783a8-01d6-4762-b63d-89f29049c440 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:36.638850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529783a8-01d6-4762-b63d-89f29049c440 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:36.654018 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 353/681] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:36.655258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789b471-8ddc-4961-8091-85001f7a19a8 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:36.655549 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789b471-8ddc-4961-8091-85001f7a19a8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:36.655826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789b471-8ddc-4961-8091-85001f7a19a8 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:36.657227 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 354/682] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:36.661560 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789b471-8ddc-4961-8091-85001f7a19a8 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:36.667162 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c41ae9-65fe-4609-8eaf-a25efeceea57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:36.667788 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c41ae9-65fe-4609-8eaf-a25efeceea57 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:36.668074 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c41ae9-65fe-4609-8eaf-a25efeceea57 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:36.668352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c41ae9-65fe-4609-8eaf-a25efeceea57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:36.669149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c41ae9-65fe-4609-8eaf-a25efeceea57 None admin] RBAC: Adding query filter params (role.id=a66ab77a63174592a1e468d2464e8bc4, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=09b96431ac98447283960019cadde8a3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:36.669464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c41ae9-65fe-4609-8eaf-a25efeceea57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:36.670928 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c41ae9-65fe-4609-8eaf-a25efeceea57 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:36.681567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 329/683] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:15:36 2022] GET /identity/v3/role_assignments?user.id=09b96431ac98447283960019cadde8a3&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=a66ab77a63174592a1e468d2464e8bc4 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:36.681567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c41ae9-65fe-4609-8eaf-a25efeceea57 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:36.798457 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c41ae9-65fe-4609-8eaf-a25efeceea57 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:36.813936 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789b471-8ddc-4961-8091-85001f7a19a8 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:36.832947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:36.841100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad857bb-4096-497d-bc92-7b188d264604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:36.841897 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad857bb-4096-497d-bc92-7b188d264604 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:36.842247 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad857bb-4096-497d-bc92-7b188d264604 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:36.843040 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad857bb-4096-497d-bc92-7b188d264604 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:36.843921 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad857bb-4096-497d-bc92-7b188d264604 None admin] RBAC: Adding query filter params (role.id=a66ab77a63174592a1e468d2464e8bc4, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=0fecb440c0874ede9f3cf544cfed93ab) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:36.844441 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad857bb-4096-497d-bc92-7b188d264604 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:36.846305 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad857bb-4096-497d-bc92-7b188d264604 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:36.854434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 330/684] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:15:36 2022] GET /identity/v3/role_assignments?user.id=0fecb440c0874ede9f3cf544cfed93ab&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=a66ab77a63174592a1e468d2464e8bc4 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:36.855645 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f1a63-b848-4e12-af36-fbf9a0f856f2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:36.855932 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f1a63-b848-4e12-af36-fbf9a0f856f2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:36.856208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f1a63-b848-4e12-af36-fbf9a0f856f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:36.862018 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584579aa-b996-4fc8-9ac3-bdad5d86c517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:36.862996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584579aa-b996-4fc8-9ac3-bdad5d86c517 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:36.863495 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584579aa-b996-4fc8-9ac3-bdad5d86c517 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:36.863840 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584579aa-b996-4fc8-9ac3-bdad5d86c517 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:36.864729 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584579aa-b996-4fc8-9ac3-bdad5d86c517 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:36.865132 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584579aa-b996-4fc8-9ac3-bdad5d86c517 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:36.866801 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584579aa-b996-4fc8-9ac3-bdad5d86c517 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:36.876573 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-096f1a63-b848-4e12-af36-fbf9a0f856f2 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:36.878268 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-096f1a63-b848-4e12-af36-fbf9a0f856f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:36.888379 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 355/685] 10.222.0.39 () {56 vars in 1135 bytes} [Thu Jul 28 22:15:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:36.896682 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584579aa-b996-4fc8-9ac3-bdad5d86c517 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:36.932639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:36.984409 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 331/686] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:37.021156 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f1a63-b848-4e12-af36-fbf9a0f856f2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:37.035122 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584579aa-b996-4fc8-9ac3-bdad5d86c517 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:37.091591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60017fb0-6abf-42b4-9f98-f4ff1383990c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:37.092703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60017fb0-6abf-42b4-9f98-f4ff1383990c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:37.093202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60017fb0-6abf-42b4-9f98-f4ff1383990c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:37.093711 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60017fb0-6abf-42b4-9f98-f4ff1383990c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:37.167022 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f1a63-b848-4e12-af36-fbf9a0f856f2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:37.193636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60017fb0-6abf-42b4-9f98-f4ff1383990c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:37.195788 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60017fb0-6abf-42b4-9f98-f4ff1383990c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:37.199152 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:37.210569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87dd2c-655a-4e4d-aefd-df068d0f4843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:37.210793 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87dd2c-655a-4e4d-aefd-df068d0f4843 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:37.211001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87dd2c-655a-4e4d-aefd-df068d0f4843 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:37.211163 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87dd2c-655a-4e4d-aefd-df068d0f4843 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:37.229746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87dd2c-655a-4e4d-aefd-df068d0f4843 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:37.233091 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:37.236705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-8b87dd2c-655a-4e4d-aefd-df068d0f4843 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:15:37.236705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:37.236705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:37.236705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:37.236705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:37.236705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:37.236705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:37.236705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:37.236705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:37.236705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:37.236705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:37.236705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:37.236705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:37.236705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:37.236705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:37.236705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:37.237716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:37.237716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:37.237716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:37.237716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:37.237716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:37.237716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:37.237716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:37.237716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:37.237716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:37.237716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:37.237716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:37.237716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:37.237716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:37.237716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:37.237716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:37.237716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:37.238576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:37.238576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:37.238576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:37.238576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:37.238576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:37.238576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:37.238576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:37.238576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:37.238576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:37.238576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:37.238576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:15:37.238576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:37.238576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 332/687] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:15:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:37.246943 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87dd2c-655a-4e4d-aefd-df068d0f4843 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:37.338842 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 356/688] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:15:37.351532 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60017fb0-6abf-42b4-9f98-f4ff1383990c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:37.373194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87dd2c-655a-4e4d-aefd-df068d0f4843 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:37.411025 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5dd8c-5483-4e4f-bda3-3adf810d8e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:37.411947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5dd8c-5483-4e4f-bda3-3adf810d8e20 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:37.412376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5dd8c-5483-4e4f-bda3-3adf810d8e20 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:37.412828 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5dd8c-5483-4e4f-bda3-3adf810d8e20 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:37.413791 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5dd8c-5483-4e4f-bda3-3adf810d8e20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:37.414244 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5dd8c-5483-4e4f-bda3-3adf810d8e20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:37.416029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5dd8c-5483-4e4f-bda3-3adf810d8e20 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:37.425888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 333/689] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:15:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:37.436789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5dd8c-5483-4e4f-bda3-3adf810d8e20 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:37.473633 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60017fb0-6abf-42b4-9f98-f4ff1383990c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:37.493277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:37.508315 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20eef95-2ae9-4564-84ca-146d8754a9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:37.509192 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20eef95-2ae9-4564-84ca-146d8754a9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:37.509858 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20eef95-2ae9-4564-84ca-146d8754a9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:37.510369 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20eef95-2ae9-4564-84ca-146d8754a9a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:37.514384 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:37.522869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20eef95-2ae9-4564-84ca-146d8754a9a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:37.525329 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20eef95-2ae9-4564-84ca-146d8754a9a9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:37.541850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-c20eef95-2ae9-4564-84ca-146d8754a9a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:37.541850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:37.541850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:37.541850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:37.541850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:37.541850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:37.541850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:37.541850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:37.541850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:37.541850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:37.541850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:37.541850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:37.541850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:37.541850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:37.541850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:37.542786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:37.542786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:37.542786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:37.542786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:37.542786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:37.542786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:37.542786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:37.542786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:37.542786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:37.542786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:37.542786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:37.542786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:37.542786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:37.542786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:37.542786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:37.542786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:37.542786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:37.543686 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:37.543686 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:37.543686 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:37.543686 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:37.546009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 357/690] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:15:37.551571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20eef95-2ae9-4564-84ca-146d8754a9a9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:37.602061 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5dd8c-5483-4e4f-bda3-3adf810d8e20 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:37.641225 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391fac4b-e148-4048-8fb8-195134bcb62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:37.641902 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fac4b-e148-4048-8fb8-195134bcb62d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:37.642184 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fac4b-e148-4048-8fb8-195134bcb62d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:37.642459 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391fac4b-e148-4048-8fb8-195134bcb62d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:37.654780 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391fac4b-e148-4048-8fb8-195134bcb62d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:37.656246 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391fac4b-e148-4048-8fb8-195134bcb62d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:37.661030 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20eef95-2ae9-4564-84ca-146d8754a9a9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:37.672415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-391fac4b-e148-4048-8fb8-195134bcb62d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:37.672415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:37.672415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:37.672415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:37.672415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:37.672415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:37.672415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:37.672415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:37.672415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:37.672415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:37.672415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:37.672415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:37.672415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:37.672415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:37.672415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:37.673382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:37.673382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:37.673382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:37.673382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:37.673382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:37.673382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:37.673382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:37.673382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:37.673382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:37.673382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:37.673382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:37.673382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:37.673382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:37.673382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:37.673382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:37.673382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:37.673382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:37.674331 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:37.674331 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:37.674331 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:37.674331 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:37.674331 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 334/691] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:37.675095 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d714d3a-2937-4609-b376-0c2c17be610a None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:37.675381 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d714d3a-2937-4609-b376-0c2c17be610a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:37.675655 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d714d3a-2937-4609-b376-0c2c17be610a None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:37.681817 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 335/692] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:37.682596 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d714d3a-2937-4609-b376-0c2c17be610a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:37.702117 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331403f0-845a-4c08-8040-cbf54b04ee77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:37.703072 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331403f0-845a-4c08-8040-cbf54b04ee77 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:37.703616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331403f0-845a-4c08-8040-cbf54b04ee77 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:37.704116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331403f0-845a-4c08-8040-cbf54b04ee77 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:37.705086 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331403f0-845a-4c08-8040-cbf54b04ee77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:37.705639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331403f0-845a-4c08-8040-cbf54b04ee77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:37.707322 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331403f0-845a-4c08-8040-cbf54b04ee77 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:37.721330 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 358/693] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:37.727220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d66f7b-9bf0-4fa3-b63a-aa8679f7f0f3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:37.728346 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d66f7b-9bf0-4fa3-b63a-aa8679f7f0f3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:37.728851 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d66f7b-9bf0-4fa3-b63a-aa8679f7f0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:37.747538 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-08d66f7b-9bf0-4fa3-b63a-aa8679f7f0f3 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:37.749110 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-08d66f7b-9bf0-4fa3-b63a-aa8679f7f0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:37.777391 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:37.794395 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d714d3a-2937-4609-b376-0c2c17be610a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:37.805881 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:37.822815 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 359/694] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:37.824022 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d66f7b-9bf0-4fa3-b63a-aa8679f7f0f3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:37.834148 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ed95f-293b-430b-959b-e6b8d1b22cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:37.834775 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ed95f-293b-430b-959b-e6b8d1b22cbb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:37.835056 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ed95f-293b-430b-959b-e6b8d1b22cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:37.835758 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ed95f-293b-430b-959b-e6b8d1b22cbb None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:37.836207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ed95f-293b-430b-959b-e6b8d1b22cbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:37.836516 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ed95f-293b-430b-959b-e6b8d1b22cbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:37.839046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ed95f-293b-430b-959b-e6b8d1b22cbb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:37.853516 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 336/695] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:37.859147 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ed95f-293b-430b-959b-e6b8d1b22cbb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:37.930833 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d66f7b-9bf0-4fa3-b63a-aa8679f7f0f3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:37.967484 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d662f6-8256-4a87-9948-55836bbc4884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:37.968601 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d662f6-8256-4a87-9948-55836bbc4884 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:37.969105 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d662f6-8256-4a87-9948-55836bbc4884 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:37.969612 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d662f6-8256-4a87-9948-55836bbc4884 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:37.974901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ed95f-293b-430b-959b-e6b8d1b22cbb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:37.977156 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d662f6-8256-4a87-9948-55836bbc4884 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:37.979528 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d662f6-8256-4a87-9948-55836bbc4884 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:37.992388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-a7d662f6-8256-4a87-9948-55836bbc4884 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:37.992388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:37.992388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:37.992388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:37.992388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:37.992388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:37.992388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:37.992388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:37.992388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:37.992388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:37.992388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:37.992388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:37.992388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:37.992388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:37.992388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:37.993388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:37.993388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:37.993388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:37.993388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:37.993388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:37.993388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:37.993388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:37.993388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:37.993388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:37.993388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:37.993388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:37.993388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:37.993388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:37.993388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:37.993388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:37.993388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:37.993388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:37.994320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:37.994320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:37.994320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:37.994320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:37.994320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:37.994320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:37.994320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:37.994320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:38.001389 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 360/696] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:38.002665 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d662f6-8256-4a87-9948-55836bbc4884 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:38.014006 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabaf70-e917-4df8-b0b5-c00fb20e6123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:38.014919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabaf70-e917-4df8-b0b5-c00fb20e6123 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:38.015199 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabaf70-e917-4df8-b0b5-c00fb20e6123 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:38.015508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaabaf70-e917-4df8-b0b5-c00fb20e6123 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:38.016702 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaabaf70-e917-4df8-b0b5-c00fb20e6123 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:38.018248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaabaf70-e917-4df8-b0b5-c00fb20e6123 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:38.033149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-aaabaf70-e917-4df8-b0b5-c00fb20e6123 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 22:15:38.033149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:38.033149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:15:38.033149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:15:38.033149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:15:38.033149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:15:38.033149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:15:38.033149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:38.033149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 22:15:38.033149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:38.033149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:15:38.033149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:38.033149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:38.033149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:38.033149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:38.033149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:38.034390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:38.034390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:38.034390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:38.034390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:38.034390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:38.034390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:38.034390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:38.034390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:15:38.034390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:15:38.034390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:15:38.034390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:15:38.034390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:38.034390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:38.034390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:38.034390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:38.034390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:38.035255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:38.035255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:38.035255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:38.035255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:38.035255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:38.035255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:38.035255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:38.035255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:38.035255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:38.035255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:38.035255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:38.035255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:15:38.035255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:38.035255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 22:15:38.035255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:38.035255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 337/697] 10.222.0.39 () {56 vars in 1129 bytes} [Thu Jul 28 22:15:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:38.036125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabaf70-e917-4df8-b0b5-c00fb20e6123 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:38.037398 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:38.077977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:38.130367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d662f6-8256-4a87-9948-55836bbc4884 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:38.178853 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaabaf70-e917-4df8-b0b5-c00fb20e6123 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:38.182553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567d5f3-f0a2-4bc2-9e58-4ad99af6f791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:38.184188 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567d5f3-f0a2-4bc2-9e58-4ad99af6f791 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:38.184571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567d5f3-f0a2-4bc2-9e58-4ad99af6f791 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:38.184967 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567d5f3-f0a2-4bc2-9e58-4ad99af6f791 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:38.194913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567d5f3-f0a2-4bc2-9e58-4ad99af6f791 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:38.197938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567d5f3-f0a2-4bc2-9e58-4ad99af6f791 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:38.214572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-5567d5f3-f0a2-4bc2-9e58-4ad99af6f791 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 22:15:38.214572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:38.214572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:38.214572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:38.214572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:38.214572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:38.214572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:38.214572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:38.214572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:38.214572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:38.214572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:38.214572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:38.214572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:38.214572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:38.214572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:38.214572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:38.215741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:38.215741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:38.215741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:38.215741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:38.215741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:38.215741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:38.215741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:38.215741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:38.215741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:38.215741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:38.215741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:38.215741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:38.215741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:38.215741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:38.215741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:38.215741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:38.215741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:38.215741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:38.216948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:38.216948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:38.216948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:38.216948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:38.216948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:38.216948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:38.216948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:38.216948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:38.216948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 22:15:38.216948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:38.222474 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 361/698] 10.222.0.39 () {56 vars in 1093 bytes} [Thu Jul 28 22:15:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:38.222474 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567d5f3-f0a2-4bc2-9e58-4ad99af6f791 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:38.225334 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a57d69-df7e-48ff-9b8a-163380f06684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:38.226209 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a57d69-df7e-48ff-9b8a-163380f06684 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:38.226702 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a57d69-df7e-48ff-9b8a-163380f06684 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:38.227167 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a57d69-df7e-48ff-9b8a-163380f06684 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:38.228503 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a57d69-df7e-48ff-9b8a-163380f06684 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:38.229040 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a57d69-df7e-48ff-9b8a-163380f06684 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:38.230813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a57d69-df7e-48ff-9b8a-163380f06684 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:38.244912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 338/699] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:38.246098 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a57d69-df7e-48ff-9b8a-163380f06684 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:38.326908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:38.338138 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:38.355065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567d5f3-f0a2-4bc2-9e58-4ad99af6f791 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:38.357559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a57d69-df7e-48ff-9b8a-163380f06684 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:38.396107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fd519-42fd-44a6-abbb-a151158424cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:38.397061 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fd519-42fd-44a6-abbb-a151158424cc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:38.397617 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fd519-42fd-44a6-abbb-a151158424cc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:38.398101 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fd519-42fd-44a6-abbb-a151158424cc None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:38.398222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0e780-275d-4509-aec6-3d92612eb047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:38.398754 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0e780-275d-4509-aec6-3d92612eb047 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:38.399039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0e780-275d-4509-aec6-3d92612eb047 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:38.399388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b0e780-275d-4509-aec6-3d92612eb047 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:38.399994 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13fd519-42fd-44a6-abbb-a151158424cc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:38.400100 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0e780-275d-4509-aec6-3d92612eb047 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:38.400364 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0e780-275d-4509-aec6-3d92612eb047 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:38.400779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13fd519-42fd-44a6-abbb-a151158424cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:38.401759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b0e780-275d-4509-aec6-3d92612eb047 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:38.402874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13fd519-42fd-44a6-abbb-a151158424cc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:38.407766 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 362/700] 10.222.0.39 () {56 vars in 1124 bytes} [Thu Jul 28 22:15:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:38.408572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0e780-275d-4509-aec6-3d92612eb047 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:38.423034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 339/701] 10.222.0.39 () {56 vars in 1133 bytes} [Thu Jul 28 22:15:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:38.424269 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fd519-42fd-44a6-abbb-a151158424cc None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:38.547724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fd519-42fd-44a6-abbb-a151158424cc None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:38.548645 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b0e780-275d-4509-aec6-3d92612eb047 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:38.584201 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6ab7c-bb79-495c-84fd-6087ecb9dce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:38.585163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6ab7c-bb79-495c-84fd-6087ecb9dce1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:38.585709 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6ab7c-bb79-495c-84fd-6087ecb9dce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:38.586304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6ab7c-bb79-495c-84fd-6087ecb9dce1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:38.587897 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6ab7c-bb79-495c-84fd-6087ecb9dce1 None admin] RBAC: Adding query filter params (role.id=a66ab77a63174592a1e468d2464e8bc4, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=e3c9107872f147f6ab8698b67bb545af) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:38.588417 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6ab7c-bb79-495c-84fd-6087ecb9dce1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:38.590103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6ab7c-bb79-495c-84fd-6087ecb9dce1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:38.591266 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91447f-ad03-4bff-bf98-327aa036df74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:38.591898 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91447f-ad03-4bff-bf98-327aa036df74 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:38.592180 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91447f-ad03-4bff-bf98-327aa036df74 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:38.592453 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91447f-ad03-4bff-bf98-327aa036df74 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:38.593311 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91447f-ad03-4bff-bf98-327aa036df74 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:38.593641 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91447f-ad03-4bff-bf98-327aa036df74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:38.595463 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91447f-ad03-4bff-bf98-327aa036df74 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:38.598242 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 363/702] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:15:38 2022] GET /identity/v3/role_assignments?user.id=e3c9107872f147f6ab8698b67bb545af&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=a66ab77a63174592a1e468d2464e8bc4 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:38.599256 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6ab7c-bb79-495c-84fd-6087ecb9dce1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:38.601919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 340/703] 10.222.0.39 () {56 vars in 1124 bytes} [Thu Jul 28 22:15:38 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:38.618665 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:38.728302 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6ab7c-bb79-495c-84fd-6087ecb9dce1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:38.772177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f759a896-67cb-49c9-abd2-4b21678adccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:38.773288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759a896-67cb-49c9-abd2-4b21678adccc None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:38.773813 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759a896-67cb-49c9-abd2-4b21678adccc None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:38.774854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f759a896-67cb-49c9-abd2-4b21678adccc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:38.863547 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759a896-67cb-49c9-abd2-4b21678adccc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:38.865033 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f759a896-67cb-49c9-abd2-4b21678adccc None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:38.878520 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:38.975773 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 364/704] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 28 22:15:38.982635 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91447f-ad03-4bff-bf98-327aa036df74 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:38.994907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:39.057736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b6932-49cf-452a-974f-e870658049ae None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:39.058205 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b6932-49cf-452a-974f-e870658049ae None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:39.058499 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669b6932-49cf-452a-974f-e870658049ae None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:39.060401 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 365/705] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:39.065288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9427963-2620-4f09-9492-c01bc9fc6715 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:39.065602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9427963-2620-4f09-9492-c01bc9fc6715 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:39.065883 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9427963-2620-4f09-9492-c01bc9fc6715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:39.086262 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f9427963-2620-4f09-9492-c01bc9fc6715 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:39.088092 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f9427963-2620-4f09-9492-c01bc9fc6715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:39.098041 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91447f-ad03-4bff-bf98-327aa036df74 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:39.141052 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2921c2-042e-417b-b6c1-501f8be188a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:39.142055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2921c2-042e-417b-b6c1-501f8be188a1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:39.142519 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2921c2-042e-417b-b6c1-501f8be188a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:39.142967 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2921c2-042e-417b-b6c1-501f8be188a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:39.160928 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2921c2-042e-417b-b6c1-501f8be188a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:39.162436 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2921c2-042e-417b-b6c1-501f8be188a1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:39.181039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-dd2921c2-042e-417b-b6c1-501f8be188a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:39.181039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:39.181039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:39.181039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:39.181039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:39.181039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:39.181039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:39.181039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:39.181039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:39.181039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:39.181039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:39.181039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:39.181039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:39.181039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:39.181039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:39.182690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:39.182690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:39.182690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:39.182690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:39.182690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:39.182690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:39.182690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:39.182690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:39.182690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:39.182690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:39.182690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:39.182690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:39.182690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:39.182690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:39.182690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:39.182690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:39.182690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:39.183642 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:39.183642 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:39.183642 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:39.183642 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:39.183642 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 341/706] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:39.189240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2921c2-042e-417b-b6c1-501f8be188a1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:39.190173 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 366/707] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:15:39.218150 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9427963-2620-4f09-9492-c01bc9fc6715 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:39.265549 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:39.290005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:39.319379 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2921c2-042e-417b-b6c1-501f8be188a1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:39.360817 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6e276-0760-4f63-a5eb-7824eb43f093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:39.361844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6e276-0760-4f63-a5eb-7824eb43f093 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:39.362349 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6e276-0760-4f63-a5eb-7824eb43f093 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:39.362871 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e6e276-0760-4f63-a5eb-7824eb43f093 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:39.364227 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e6e276-0760-4f63-a5eb-7824eb43f093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:39.365055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e6e276-0760-4f63-a5eb-7824eb43f093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:39.366759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e6e276-0760-4f63-a5eb-7824eb43f093 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:39.396358 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 342/708] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:39.399282 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6e276-0760-4f63-a5eb-7824eb43f093 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:39.417335 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9427963-2620-4f09-9492-c01bc9fc6715 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:39.491361 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca03b5a-3324-423e-8f9e-c7e9dfc9c0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:39.492019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca03b5a-3324-423e-8f9e-c7e9dfc9c0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:39.492299 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca03b5a-3324-423e-8f9e-c7e9dfc9c0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:39.492570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca03b5a-3324-423e-8f9e-c7e9dfc9c0a6 None admin] PATH_INFO: `/v3/services/network` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:39.493504 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca03b5a-3324-423e-8f9e-c7e9dfc9c0a6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:39.494910 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca03b5a-3324-423e-8f9e-c7e9dfc9c0a6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:39.519224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e6e276-0760-4f63-a5eb-7824eb43f093 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:39.524023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-0ca03b5a-3324-423e-8f9e-c7e9dfc9c0a6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 22:15:39.524023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:39.524023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:15:39.524023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:15:39.524023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:15:39.524023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:15:39.524023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:15:39.524023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:39.524023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 22:15:39.524023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:39.524023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:15:39.524023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:39.524023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:39.524023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:39.524023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:39.524023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:39.524023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:39.525304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:39.525304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:39.525304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:39.525304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:39.525304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:39.525304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:39.525304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:15:39.525304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:15:39.525304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:15:39.525304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:15:39.525304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:39.525304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:39.525304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:39.525304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:39.525304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:39.525304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:39.526198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:39.526198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:39.526198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:39.526198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:39.526198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:39.526198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:39.526198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:39.526198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:39.526198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:39.526198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:39.526198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:15:39.526198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:39.526198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 22:15:39.526198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:39.533973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 367/709] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:39 2022] GET /identity/v3/services/network => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:15:39.547520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:39.550566 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca03b5a-3324-423e-8f9e-c7e9dfc9c0a6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:39.572899 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9728ee9-89f1-4e48-bd81-0b879a0fe0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:39.573565 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9728ee9-89f1-4e48-bd81-0b879a0fe0bb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:39.573848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9728ee9-89f1-4e48-bd81-0b879a0fe0bb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:39.574119 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9728ee9-89f1-4e48-bd81-0b879a0fe0bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:39.588172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9728ee9-89f1-4e48-bd81-0b879a0fe0bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:39.591038 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9728ee9-89f1-4e48-bd81-0b879a0fe0bb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:39.609865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-d9728ee9-89f1-4e48-bd81-0b879a0fe0bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:39.609865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:39.609865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:39.609865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:39.609865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:39.609865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:39.609865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:39.609865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:39.609865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:39.609865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:39.609865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:39.609865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:39.609865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:39.609865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:39.609865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:39.610822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:39.610822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:39.610822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:39.610822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:39.610822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:39.610822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:39.610822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:39.610822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:39.610822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:39.610822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:39.610822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:39.610822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:39.610822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:39.610822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:39.610822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:39.610822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:39.610822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:39.611717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:39.611717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:39.611717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:39.611717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:39.611717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:39.611717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:39.611717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:39.611717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:39.614511 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 343/710] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:39.620742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f11ef-349f-4b63-9503-809e280a1155 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:39.620742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f11ef-349f-4b63-9503-809e280a1155 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:39.620742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f11ef-349f-4b63-9503-809e280a1155 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:39.620742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 344/711] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:39.626913 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f11ef-349f-4b63-9503-809e280a1155 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:39.685289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca03b5a-3324-423e-8f9e-c7e9dfc9c0a6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:39.688701 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:39.731830 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1915b6-e28f-4e90-96ec-daf90494f706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:39.732862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1915b6-e28f-4e90-96ec-daf90494f706 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:39.733252 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1915b6-e28f-4e90-96ec-daf90494f706 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:39.733664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1915b6-e28f-4e90-96ec-daf90494f706 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:39.734679 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1915b6-e28f-4e90-96ec-daf90494f706 None admin] RBAC: Adding query filter params (name=network) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:39.735103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1915b6-e28f-4e90-96ec-daf90494f706 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:39.737012 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1915b6-e28f-4e90-96ec-daf90494f706 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:39.745605 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 368/712] 10.222.0.39 () {56 vars in 1112 bytes} [Thu Jul 28 22:15:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:39.747159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454cdf1-cca2-453b-8844-a0bf78785410 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:39.747590 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454cdf1-cca2-453b-8844-a0bf78785410 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:39.747996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454cdf1-cca2-453b-8844-a0bf78785410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:39.760865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f11ef-349f-4b63-9503-809e280a1155 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:39.769400 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c454cdf1-cca2-453b-8844-a0bf78785410 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:39.770824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c454cdf1-cca2-453b-8844-a0bf78785410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:39.806745 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818b5d9-785f-47cc-ba7d-90dc3dfa2d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:39.807715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818b5d9-785f-47cc-ba7d-90dc3dfa2d64 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:39.808107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818b5d9-785f-47cc-ba7d-90dc3dfa2d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:39.808530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9818b5d9-785f-47cc-ba7d-90dc3dfa2d64 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:39.809393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818b5d9-785f-47cc-ba7d-90dc3dfa2d64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:39.809819 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818b5d9-785f-47cc-ba7d-90dc3dfa2d64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:39.812329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9818b5d9-785f-47cc-ba7d-90dc3dfa2d64 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:39.827364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 345/713] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:39.828610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818b5d9-785f-47cc-ba7d-90dc3dfa2d64 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:39.853046 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 369/714] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:39.854288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454cdf1-cca2-453b-8844-a0bf78785410 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:39.875885 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:39.958561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9818b5d9-785f-47cc-ba7d-90dc3dfa2d64 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:39.993357 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:39.999341 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454cdf1-cca2-453b-8844-a0bf78785410 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:40.008369 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb92a5-55a8-4ca5-8c65-5da90784de13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:40.010217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb92a5-55a8-4ca5-8c65-5da90784de13 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:40.010567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb92a5-55a8-4ca5-8c65-5da90784de13 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:40.010909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eb92a5-55a8-4ca5-8c65-5da90784de13 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:40.011684 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb92a5-55a8-4ca5-8c65-5da90784de13 None admin] RBAC: Adding query filter params (type=network) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:40.012805 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb92a5-55a8-4ca5-8c65-5da90784de13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:40.014638 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eb92a5-55a8-4ca5-8c65-5da90784de13 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:40.029566 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 346/715] 10.222.0.39 () {56 vars in 1112 bytes} [Thu Jul 28 22:15:39 2022] GET /identity/v3/services?type=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:40.029566 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb92a5-55a8-4ca5-8c65-5da90784de13 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:40.070470 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc0854-ac46-4168-ba28-09b4175cde20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:40.071639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc0854-ac46-4168-ba28-09b4175cde20 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:40.071916 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc0854-ac46-4168-ba28-09b4175cde20 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:40.072187 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cc0854-ac46-4168-ba28-09b4175cde20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:40.073156 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc0854-ac46-4168-ba28-09b4175cde20 None admin] RBAC: Adding query filter params (role.id=aded7bd43e974686b3c23bd9d4255b24, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=77efe8046ea0468cb0c7c58c24fc4e75) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:40.073464 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc0854-ac46-4168-ba28-09b4175cde20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:40.078822 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cc0854-ac46-4168-ba28-09b4175cde20 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:40.093247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 370/716] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:15:39 2022] GET /identity/v3/role_assignments?user.id=77efe8046ea0468cb0c7c58c24fc4e75&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=aded7bd43e974686b3c23bd9d4255b24 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:40.233978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eb92a5-55a8-4ca5-8c65-5da90784de13 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:40.235626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:40.277345 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5615f-793a-424f-a3f5-20ab0a43755e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:40.278046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5615f-793a-424f-a3f5-20ab0a43755e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:40.278335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5615f-793a-424f-a3f5-20ab0a43755e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:40.278610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5615f-793a-424f-a3f5-20ab0a43755e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:40.292175 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5615f-793a-424f-a3f5-20ab0a43755e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:40.293759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5615f-793a-424f-a3f5-20ab0a43755e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:40.309977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-94c5615f-793a-424f-a3f5-20ab0a43755e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:40.309977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:40.309977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:40.309977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:40.309977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:40.309977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:40.309977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:40.309977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:40.309977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:40.309977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:40.309977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:40.309977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:40.309977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:40.309977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:40.309977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:40.310955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:40.310955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:40.310955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:40.310955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:40.310955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:40.310955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:40.310955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:40.310955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:40.310955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:40.310955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:40.310955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:40.310955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:40.310955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:40.310955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:40.310955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:40.310955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:40.310955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:40.311886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:40.311886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:40.311886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:40.311886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:40.311886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 347/717] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:40.317044 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc0854-ac46-4168-ba28-09b4175cde20 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:40.436256 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cc0854-ac46-4168-ba28-09b4175cde20 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:40.483166 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55799d2-51cb-47ca-ae3c-f450e87f9eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:40.484276 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55799d2-51cb-47ca-ae3c-f450e87f9eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:40.484711 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55799d2-51cb-47ca-ae3c-f450e87f9eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:40.485178 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55799d2-51cb-47ca-ae3c-f450e87f9eb4 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:40.485946 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55799d2-51cb-47ca-ae3c-f450e87f9eb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:40.486254 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55799d2-51cb-47ca-ae3c-f450e87f9eb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:40.487637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55799d2-51cb-47ca-ae3c-f450e87f9eb4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:40.500868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 371/718] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:40.506860 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5615f-793a-424f-a3f5-20ab0a43755e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:40.630821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5615f-793a-424f-a3f5-20ab0a43755e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:40.682503 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ead092-0c6b-44d4-a757-c5ecb44f6bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:40.683184 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ead092-0c6b-44d4-a757-c5ecb44f6bdf None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:40.683468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ead092-0c6b-44d4-a757-c5ecb44f6bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:40.683791 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ead092-0c6b-44d4-a757-c5ecb44f6bdf None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:40.684490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ead092-0c6b-44d4-a757-c5ecb44f6bdf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:40.685969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ead092-0c6b-44d4-a757-c5ecb44f6bdf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:40.725803 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 348/719] 10.222.0.39 () {60 vars in 1132 bytes} [Thu Jul 28 22:15:40 2022] POST /identity/v3/users => generated 254 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:15:41.054061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76aff0-ecac-496d-a55f-afbda7e8d8fb None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:41.054458 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76aff0-ecac-496d-a55f-afbda7e8d8fb None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:41.054635 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76aff0-ecac-496d-a55f-afbda7e8d8fb None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:41.056822 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 372/720] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:41.062558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a0512-f277-4d9f-b0f1-c1943fe5e4af None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:41.062846 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a0512-f277-4d9f-b0f1-c1943fe5e4af None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:41.063171 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a0512-f277-4d9f-b0f1-c1943fe5e4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:41.081929 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f90a0512-f277-4d9f-b0f1-c1943fe5e4af None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:41.083115 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f90a0512-f277-4d9f-b0f1-c1943fe5e4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:41.086061 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:41.169173 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 349/721] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:41.194843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76aff0-ecac-496d-a55f-afbda7e8d8fb None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:41.224059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:41.315228 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76aff0-ecac-496d-a55f-afbda7e8d8fb None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:41.358708 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf111e-ffb3-495d-a388-2d3aa1f5fe50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:41.359330 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf111e-ffb3-495d-a388-2d3aa1f5fe50 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:41.359610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf111e-ffb3-495d-a388-2d3aa1f5fe50 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:41.359882 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf111e-ffb3-495d-a388-2d3aa1f5fe50 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:41.360585 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf111e-ffb3-495d-a388-2d3aa1f5fe50 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:41.361980 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf111e-ffb3-495d-a388-2d3aa1f5fe50 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:41.386406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c5b7c-8063-4435-b898-2be861a4f190 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:41.386937 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c5b7c-8063-4435-b898-2be861a4f190 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:41.387413 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c5b7c-8063-4435-b898-2be861a4f190 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:41.391067 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 350/722] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:41.395616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49903158-9fc6-49fa-ab91-5bd2c017bb19 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:41.395905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49903158-9fc6-49fa-ab91-5bd2c017bb19 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:41.396182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49903158-9fc6-49fa-ab91-5bd2c017bb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:41.396777 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 373/723] 10.222.0.39 () {60 vars in 1141 bytes} [Thu Jul 28 22:15:41 2022] POST /identity/v3/services => generated 251 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:15:41.426758 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-49903158-9fc6-49fa-ab91-5bd2c017bb19 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:41.427996 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-49903158-9fc6-49fa-ab91-5bd2c017bb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:41.638287 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 351/724] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:41 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 28 22:15:41.662030 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf111e-ffb3-495d-a388-2d3aa1f5fe50 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:41.751334 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:41.772552 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf111e-ffb3-495d-a388-2d3aa1f5fe50 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:41.810866 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b59af-7091-48f6-82c3-3dec88694324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:41.811569 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b59af-7091-48f6-82c3-3dec88694324 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:41.811853 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b59af-7091-48f6-82c3-3dec88694324 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:41.812180 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b59af-7091-48f6-82c3-3dec88694324 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:41.823313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b59af-7091-48f6-82c3-3dec88694324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:41.828468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-672b59af-7091-48f6-82c3-3dec88694324 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:15:41.828468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:41.828468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:41.828468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:41.828468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:41.828468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:41.828468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:41.828468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:41.828468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:41.828468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:41.828468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:41.828468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:41.828468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:41.828468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:41.828468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:41.828468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:41.829384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:41.829384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:41.829384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:41.829384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:41.829384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:41.829384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:41.829384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:41.829384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:41.829384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:41.829384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:41.829384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:41.829384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:41.829384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:41.829384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:41.829384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:41.829384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:41.830269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:41.830269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:41.830269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:41.830269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:41.830269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:41.830269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:41.830269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:41.830269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:41.830269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:41.830269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:41.830269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:15:41.830269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:41.833655 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 374/725] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:15:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:41.835936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49903158-9fc6-49fa-ab91-5bd2c017bb19 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:41.911204 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:41.956825 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49903158-9fc6-49fa-ab91-5bd2c017bb19 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:41.991692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbbc3e-cf9c-4752-82fa-be174014cf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:41.992318 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbbc3e-cf9c-4752-82fa-be174014cf43 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:41.992659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbbc3e-cf9c-4752-82fa-be174014cf43 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:41.993453 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fbbc3e-cf9c-4752-82fa-be174014cf43 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:41.994211 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fbbc3e-cf9c-4752-82fa-be174014cf43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:41.994517 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fbbc3e-cf9c-4752-82fa-be174014cf43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:41.995882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fbbc3e-cf9c-4752-82fa-be174014cf43 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:42.009686 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 352/726] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:15:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:42.011086 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b59af-7091-48f6-82c3-3dec88694324 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:42.116416 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b59af-7091-48f6-82c3-3dec88694324 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:42.153672 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2faa5a-973e-486d-8e60-8187fdb83da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:42.157144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2faa5a-973e-486d-8e60-8187fdb83da2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:42.157144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2faa5a-973e-486d-8e60-8187fdb83da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:42.157144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2faa5a-973e-486d-8e60-8187fdb83da2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:42.166604 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2faa5a-973e-486d-8e60-8187fdb83da2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:42.168010 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2faa5a-973e-486d-8e60-8187fdb83da2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:42.184951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ab2faa5a-973e-486d-8e60-8187fdb83da2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:42.184951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:42.184951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:42.184951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:42.184951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:42.184951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:42.184951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:42.184951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:42.184951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:42.184951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:42.184951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:42.184951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:42.184951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:42.184951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:42.184951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:42.185927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:42.185927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:42.185927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:42.185927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:42.185927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:42.185927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:42.185927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:42.185927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:42.185927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:42.185927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:42.185927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:42.185927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:42.185927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:42.185927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:42.185927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:42.185927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:42.185927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:42.186827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:42.186827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:42.186827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:42.186827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:42.189877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:42.190925 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 375/727] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:42.192266 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbbc3e-cf9c-4752-82fa-be174014cf43 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:42.296386 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fbbc3e-cf9c-4752-82fa-be174014cf43 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:42.324444 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:42.332738 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448114e-8be6-4580-b86d-d2d689d91d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:42.333362 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448114e-8be6-4580-b86d-d2d689d91d20 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:42.333680 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448114e-8be6-4580-b86d-d2d689d91d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:42.333958 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448114e-8be6-4580-b86d-d2d689d91d20 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:42.334658 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448114e-8be6-4580-b86d-d2d689d91d20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:42.334964 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448114e-8be6-4580-b86d-d2d689d91d20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:42.337342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448114e-8be6-4580-b86d-d2d689d91d20 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:42.349560 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 353/728] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:42.355846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2faa5a-973e-486d-8e60-8187fdb83da2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:42.463550 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2faa5a-973e-486d-8e60-8187fdb83da2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:42.499950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7442ba1d-57cb-4987-a49d-023b6757330f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:42.500626 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442ba1d-57cb-4987-a49d-023b6757330f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:42.500913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442ba1d-57cb-4987-a49d-023b6757330f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:42.501193 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7442ba1d-57cb-4987-a49d-023b6757330f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:42.509949 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442ba1d-57cb-4987-a49d-023b6757330f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:42.511436 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7442ba1d-57cb-4987-a49d-023b6757330f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:42.524515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-7442ba1d-57cb-4987-a49d-023b6757330f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 22:15:42.524515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:42.524515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:42.524515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:42.524515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:42.524515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:42.524515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:42.524515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:42.524515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:42.524515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:42.524515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:42.524515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:42.524515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:42.524515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:42.524515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:42.524515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:42.525593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:42.525593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:42.525593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:42.525593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:42.525593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:42.525593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:42.525593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:42.525593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:42.525593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:42.525593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:42.525593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:42.525593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:42.525593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:42.525593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:42.525593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:42.525593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:42.525593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:42.525593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:42.526583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:42.526583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:42.526583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:42.526583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:42.526583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:42.526583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:42.526583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:42.526583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:42.526583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 22:15:42.526583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:42.529637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 376/729] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:15:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:42.531848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448114e-8be6-4580-b86d-d2d689d91d20 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:42.637107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448114e-8be6-4580-b86d-d2d689d91d20 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:42.673335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4487e-3ce6-40c3-96e6-e1e1dc269b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:42.673990 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4487e-3ce6-40c3-96e6-e1e1dc269b5f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:42.674273 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4487e-3ce6-40c3-96e6-e1e1dc269b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:42.674550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab4487e-3ce6-40c3-96e6-e1e1dc269b5f None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:42.675359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4487e-3ce6-40c3-96e6-e1e1dc269b5f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:42.675668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4487e-3ce6-40c3-96e6-e1e1dc269b5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:42.677096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab4487e-3ce6-40c3-96e6-e1e1dc269b5f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:42.691426 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093c7a8-a24e-468a-952d-07f72edb7e86 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:42.691914 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093c7a8-a24e-468a-952d-07f72edb7e86 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:42.691991 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093c7a8-a24e-468a-952d-07f72edb7e86 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:42.693377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 377/730] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:42.698532 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05531c7d-9ce0-4cd0-9ac6-d8ac3cb0a1b3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:42.698820 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05531c7d-9ce0-4cd0-9ac6-d8ac3cb0a1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:42.699156 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05531c7d-9ce0-4cd0-9ac6-d8ac3cb0a1b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:42.703632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 354/731] 10.222.0.39 () {56 vars in 1135 bytes} [Thu Jul 28 22:15:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:42.704798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4487e-3ce6-40c3-96e6-e1e1dc269b5f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:42.735960 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-05531c7d-9ce0-4cd0-9ac6-d8ac3cb0a1b3 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:42.737256 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-05531c7d-9ce0-4cd0-9ac6-d8ac3cb0a1b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:42.769899 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:42.813131 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab4487e-3ce6-40c3-96e6-e1e1dc269b5f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:42.828156 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:42.857557 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 378/732] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:42 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:42.857557 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12523879-84ee-458d-8414-554dbf6705d7 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:42.857557 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12523879-84ee-458d-8414-554dbf6705d7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:42.857557 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12523879-84ee-458d-8414-554dbf6705d7 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:42.857557 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 379/733] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:42.867546 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715539b-3a88-4719-9299-810acc8b0b10 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:42.868013 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715539b-3a88-4719-9299-810acc8b0b10 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:42.868296 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3715539b-3a88-4719-9299-810acc8b0b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:42.869269 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba9aca-8fb5-4ca1-8945-08838483eb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:42.871897 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba9aca-8fb5-4ca1-8945-08838483eb37 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:42.872184 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba9aca-8fb5-4ca1-8945-08838483eb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:42.872460 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba9aca-8fb5-4ca1-8945-08838483eb37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:42.894224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3715539b-3a88-4719-9299-810acc8b0b10 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:42.895582 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3715539b-3a88-4719-9299-810acc8b0b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:42.950716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba9aca-8fb5-4ca1-8945-08838483eb37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:42.952604 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba9aca-8fb5-4ca1-8945-08838483eb37 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:42.973752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 380/734] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:42 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 28 22:15:42.974523 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715539b-3a88-4719-9299-810acc8b0b10 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:43.004116 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:43.066219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 355/735] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:42 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 28 22:15:43.067395 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba9aca-8fb5-4ca1-8945-08838483eb37 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:43.096871 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3715539b-3a88-4719-9299-810acc8b0b10 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:43.112842 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:43.134485 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da4c3d-d4d2-480c-8d02-662a790f1361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:43.135240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da4c3d-d4d2-480c-8d02-662a790f1361 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:43.135457 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da4c3d-d4d2-480c-8d02-662a790f1361 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:43.135728 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da4c3d-d4d2-480c-8d02-662a790f1361 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:43.136437 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da4c3d-d4d2-480c-8d02-662a790f1361 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:43.137842 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da4c3d-d4d2-480c-8d02-662a790f1361 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:43.161589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 381/736] 10.222.0.39 () {60 vars in 1141 bytes} [Thu Jul 28 22:15:42 2022] POST /identity/v3/services => generated 240 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:15:43.161589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da4c3d-d4d2-480c-8d02-662a790f1361 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:43.225544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba9aca-8fb5-4ca1-8945-08838483eb37 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:43.308400 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da4c3d-d4d2-480c-8d02-662a790f1361 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:43.311101 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a98f6-d2b4-43ad-bf74-690ae806607b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:43.312083 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a98f6-d2b4-43ad-bf74-690ae806607b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:43.312860 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a98f6-d2b4-43ad-bf74-690ae806607b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:43.314171 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a98f6-d2b4-43ad-bf74-690ae806607b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:43.335640 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a98f6-d2b4-43ad-bf74-690ae806607b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:43.344892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-634a98f6-d2b4-43ad-bf74-690ae806607b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:15:43.344892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:43.344892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:43.344892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:43.344892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:43.344892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:43.344892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:43.344892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:43.344892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:43.344892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:43.344892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:43.344892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:43.344892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:43.344892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:43.344892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:43.344892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:43.345955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:43.345955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:43.345955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:43.345955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:43.345955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:43.345955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:43.345955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:43.345955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:43.345955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:43.345955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:43.345955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:43.345955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:43.345955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:43.345955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:43.345955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:43.345955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:43.346821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:43.346821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:43.346821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:43.346821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:43.346821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:43.346821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:43.346821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:43.346821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:43.346821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:43.346821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:43.346821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:15:43.346821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:43.352041 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 356/737] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:15:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:43.365553 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a98f6-d2b4-43ad-bf74-690ae806607b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:43.368584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:43.388223 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0c04c-735b-40f1-b2d0-496bbd2d0828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:43.388872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0c04c-735b-40f1-b2d0-496bbd2d0828 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:43.389404 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0c04c-735b-40f1-b2d0-496bbd2d0828 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:43.389699 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0c04c-735b-40f1-b2d0-496bbd2d0828 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:43.406347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0c04c-735b-40f1-b2d0-496bbd2d0828 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:43.407874 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0c04c-735b-40f1-b2d0-496bbd2d0828 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:43.425904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-e9b0c04c-735b-40f1-b2d0-496bbd2d0828 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:43.425904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:43.425904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:43.425904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:43.425904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:43.425904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:43.425904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:43.425904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:43.425904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:43.425904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:43.425904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:43.425904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:43.425904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:43.425904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:43.425904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:43.426930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:43.426930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:43.426930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:43.426930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:43.426930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:43.426930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:43.426930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:43.426930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:43.426930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:43.426930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:43.426930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:43.426930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:43.426930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:43.426930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:43.426930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:43.426930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:43.426930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:43.427825 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:43.427825 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:43.427825 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:43.427825 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:43.427825 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 382/738] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:43.433554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0c04c-735b-40f1-b2d0-496bbd2d0828 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:43.465905 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:43.483591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a98f6-d2b4-43ad-bf74-690ae806607b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:43.520890 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7cf78-9f0d-44a2-838c-7da48eeb0836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:43.521513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7cf78-9f0d-44a2-838c-7da48eeb0836 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:43.521925 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7cf78-9f0d-44a2-838c-7da48eeb0836 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:43.522201 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7cf78-9f0d-44a2-838c-7da48eeb0836 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:43.522981 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7cf78-9f0d-44a2-838c-7da48eeb0836 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:43.523291 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7cf78-9f0d-44a2-838c-7da48eeb0836 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:43.525649 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7cf78-9f0d-44a2-838c-7da48eeb0836 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:43.535187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 357/739] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:15:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:43.541289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7cf78-9f0d-44a2-838c-7da48eeb0836 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:43.547771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0c04c-735b-40f1-b2d0-496bbd2d0828 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:43.594783 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f008a9-3c19-43b1-9ebf-6c73cc930c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:43.596165 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f008a9-3c19-43b1-9ebf-6c73cc930c6f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:43.596883 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f008a9-3c19-43b1-9ebf-6c73cc930c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:43.597309 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f008a9-3c19-43b1-9ebf-6c73cc930c6f None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:43.598938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f008a9-3c19-43b1-9ebf-6c73cc930c6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:43.599263 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f008a9-3c19-43b1-9ebf-6c73cc930c6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:43.600664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f008a9-3c19-43b1-9ebf-6c73cc930c6f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:43.612050 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:43.615245 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 383/740] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:43.621966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbbd4c-128e-44f3-a028-5d7e827d1fef None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:43.622286 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbbd4c-128e-44f3-a028-5d7e827d1fef None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:43.622654 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbbd4c-128e-44f3-a028-5d7e827d1fef None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:43.624213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 384/741] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:43.625065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbbd4c-128e-44f3-a028-5d7e827d1fef None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:43.660502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7cf78-9f0d-44a2-838c-7da48eeb0836 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:43.697436 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490664f0-cdaa-4c17-a2cf-59ff2c2a5491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:43.698505 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490664f0-cdaa-4c17-a2cf-59ff2c2a5491 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:43.699009 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490664f0-cdaa-4c17-a2cf-59ff2c2a5491 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:43.699478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490664f0-cdaa-4c17-a2cf-59ff2c2a5491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:43.711907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490664f0-cdaa-4c17-a2cf-59ff2c2a5491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:43.713830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490664f0-cdaa-4c17-a2cf-59ff2c2a5491 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:43.716896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:43.732870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-490664f0-cdaa-4c17-a2cf-59ff2c2a5491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:43.732870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:43.732870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:43.732870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:43.732870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:43.732870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:43.732870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:43.732870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:43.732870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:43.732870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:43.732870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:43.732870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:43.732870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:43.732870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:43.732870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:43.733886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:43.733886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:43.733886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:43.733886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:43.733886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:43.733886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:43.733886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:43.733886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:43.733886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:43.733886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:43.733886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:43.733886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:43.733886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:43.733886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:43.733886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:43.733886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:43.733886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:43.734782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:43.734782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:43.734782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:43.734782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:43.735004 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbbd4c-128e-44f3-a028-5d7e827d1fef None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:43.741724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 358/742] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:43.743802 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a0bf7-07b1-43ad-bfe5-40f0d9f3ace3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:43.744314 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a0bf7-07b1-43ad-bfe5-40f0d9f3ace3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:43.744843 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1a0bf7-07b1-43ad-bfe5-40f0d9f3ace3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:43.765318 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-cb1a0bf7-07b1-43ad-bfe5-40f0d9f3ace3 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:43.766909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cb1a0bf7-07b1-43ad-bfe5-40f0d9f3ace3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:43.772919 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83c5c8-995f-472c-bf0a-f9e4472ccfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:43.773667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83c5c8-995f-472c-bf0a-f9e4472ccfb2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:43.774071 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83c5c8-995f-472c-bf0a-f9e4472ccfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:43.774377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e83c5c8-995f-472c-bf0a-f9e4472ccfb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:43.781842 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e83c5c8-995f-472c-bf0a-f9e4472ccfb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:43.783300 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e83c5c8-995f-472c-bf0a-f9e4472ccfb2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:43.796791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-2e83c5c8-995f-472c-bf0a-f9e4472ccfb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:43.796791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:43.796791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:43.796791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:43.796791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:43.796791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:43.796791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:43.796791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:43.796791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:43.796791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:43.796791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:43.796791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:43.796791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:43.796791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:43.796791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:43.797796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:43.797796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:43.797796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:43.797796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:43.797796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:43.797796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:43.797796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:43.797796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:43.797796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:43.797796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:43.797796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:43.797796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:43.797796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:43.797796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:43.797796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:43.797796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:43.797796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:43.798703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:43.798703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:43.798703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:43.798703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:43.798703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:43.798703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:43.798703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:43.798703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:43.798703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 385/743] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:43.799667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83c5c8-995f-472c-bf0a-f9e4472ccfb2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:43.848910 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 359/744] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:15:43.850167 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a0bf7-07b1-43ad-bfe5-40f0d9f3ace3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:43.902410 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:43.907406 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e83c5c8-995f-472c-bf0a-f9e4472ccfb2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:43.941964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b5390-9e52-4e57-aeaf-4d02d5abe0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:43.942588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b5390-9e52-4e57-aeaf-4d02d5abe0da None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:43.942873 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b5390-9e52-4e57-aeaf-4d02d5abe0da None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:43.943152 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b5390-9e52-4e57-aeaf-4d02d5abe0da None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:43.943855 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b5390-9e52-4e57-aeaf-4d02d5abe0da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:43.944162 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b5390-9e52-4e57-aeaf-4d02d5abe0da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:43.945555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b5390-9e52-4e57-aeaf-4d02d5abe0da None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:43.958835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 386/745] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:43.959494 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1a0bf7-07b1-43ad-bfe5-40f0d9f3ace3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:43.959698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b5390-9e52-4e57-aeaf-4d02d5abe0da None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:43.969050 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:44.004999 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8f79a-9d8c-4eb9-913a-9309f4697c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:44.006237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8f79a-9d8c-4eb9-913a-9309f4697c7c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:44.006716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8f79a-9d8c-4eb9-913a-9309f4697c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:44.007226 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8f79a-9d8c-4eb9-913a-9309f4697c7c None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:44.008357 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8f79a-9d8c-4eb9-913a-9309f4697c7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:44.008862 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8f79a-9d8c-4eb9-913a-9309f4697c7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:44.010611 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8f79a-9d8c-4eb9-913a-9309f4697c7c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:44.024314 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 360/746] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:44.025558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8f79a-9d8c-4eb9-913a-9309f4697c7c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:44.068734 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b5390-9e52-4e57-aeaf-4d02d5abe0da None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:44.105091 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa8f56-0f58-425e-920b-eaf5c68fc5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:44.105688 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa8f56-0f58-425e-920b-eaf5c68fc5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:44.106024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa8f56-0f58-425e-920b-eaf5c68fc5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:44.106312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa8f56-0f58-425e-920b-eaf5c68fc5b7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:44.117343 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa8f56-0f58-425e-920b-eaf5c68fc5b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:44.122746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-14fa8f56-0f58-425e-920b-eaf5c68fc5b7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:44.122746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:44.122746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:44.122746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:44.122746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:44.122746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:44.122746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:44.122746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:44.122746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:44.122746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:44.122746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:44.122746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:44.122746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:44.122746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:44.122746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:44.122746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:44.123696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:44.123696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:44.123696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:44.123696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:44.123696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:44.123696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:44.123696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:44.123696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:44.123696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:44.123696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:44.123696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:44.123696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:44.123696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:44.123696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:44.123696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:44.123696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:44.124533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:44.124533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:44.124533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:44.124533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:44.124533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:44.124533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:44.124533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:44.124533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:44.124533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:44.124533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:44.124533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:44.124533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:44.124533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 387/747] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:44.125230 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa8f56-0f58-425e-920b-eaf5c68fc5b7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:44.143057 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8f79a-9d8c-4eb9-913a-9309f4697c7c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:44.151026 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:44.189863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81850e7-7a69-46fd-9e32-9d62ebb639f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:44.190505 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81850e7-7a69-46fd-9e32-9d62ebb639f6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:44.190789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81850e7-7a69-46fd-9e32-9d62ebb639f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:44.191064 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81850e7-7a69-46fd-9e32-9d62ebb639f6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:44.199941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81850e7-7a69-46fd-9e32-9d62ebb639f6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:44.201381 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81850e7-7a69-46fd-9e32-9d62ebb639f6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:44.202775 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:44.213947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-d81850e7-7a69-46fd-9e32-9d62ebb639f6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 22:15:44.213947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:44.213947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:44.213947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:44.213947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:44.213947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:44.213947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:44.213947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:44.213947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:44.213947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:44.213947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:44.213947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:44.213947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:44.213947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:44.213947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:44.213947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:44.214907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:44.214907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:44.214907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:44.214907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:44.214907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:44.214907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:44.214907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:44.214907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:44.214907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:44.214907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:44.214907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:44.214907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:44.214907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:44.214907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:44.214907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:44.214907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:44.214907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:44.214907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:44.215858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:44.215858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:44.215858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:44.215858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:44.215858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:44.215858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:44.215858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:44.215858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:44.215858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 22:15:44.215858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:44.215858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 361/748] 10.222.0.39 () {56 vars in 1093 bytes} [Thu Jul 28 22:15:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:44.216448 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a1e1f-ef5c-422c-b702-ab9ad745226e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:44.216517 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a1e1f-ef5c-422c-b702-ab9ad745226e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:44.217482 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127a1e1f-ef5c-422c-b702-ab9ad745226e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:44.218163 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 362/749] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:44.218714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a1e1f-ef5c-422c-b702-ab9ad745226e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:44.248229 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa8f56-0f58-425e-920b-eaf5c68fc5b7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:44.286453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0ba47-ac47-41bb-ab1e-950ade4870c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:44.287072 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0ba47-ac47-41bb-ab1e-950ade4870c8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:44.287356 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0ba47-ac47-41bb-ab1e-950ade4870c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:44.287628 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0ba47-ac47-41bb-ab1e-950ade4870c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:44.288415 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab0ba47-ac47-41bb-ab1e-950ade4870c8 None admin] RBAC: Adding query filter params (role.id=aded7bd43e974686b3c23bd9d4255b24, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=e3c9107872f147f6ab8698b67bb545af) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:44.288725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab0ba47-ac47-41bb-ab1e-950ade4870c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:44.290836 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab0ba47-ac47-41bb-ab1e-950ade4870c8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:44.305565 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 388/750] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:15:44 2022] GET /identity/v3/role_assignments?user.id=e3c9107872f147f6ab8698b67bb545af&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=aded7bd43e974686b3c23bd9d4255b24 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:44.305565 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0ba47-ac47-41bb-ab1e-950ade4870c8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:44.328055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127a1e1f-ef5c-422c-b702-ab9ad745226e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:44.366598 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8d0ab6-4515-4630-bcad-ce28359605c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:44.367261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d0ab6-4515-4630-bcad-ce28359605c0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:44.367554 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d0ab6-4515-4630-bcad-ce28359605c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:44.368394 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8d0ab6-4515-4630-bcad-ce28359605c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:44.369071 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d0ab6-4515-4630-bcad-ce28359605c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:44.369382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d0ab6-4515-4630-bcad-ce28359605c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:44.370922 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8d0ab6-4515-4630-bcad-ce28359605c0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:44.385572 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 363/751] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:15:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:44.385572 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c8798-5620-44fb-af29-cc23f283d470 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:44.385572 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c8798-5620-44fb-af29-cc23f283d470 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:44.385572 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368c8798-5620-44fb-af29-cc23f283d470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:44.404533 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-368c8798-5620-44fb-af29-cc23f283d470 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:44.405881 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-368c8798-5620-44fb-af29-cc23f283d470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:44.434144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0ba47-ac47-41bb-ab1e-950ade4870c8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:44.441938 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:44.468689 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:44.477257 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83374e8-f24e-4512-a1ab-8cb82d523e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:44.478339 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83374e8-f24e-4512-a1ab-8cb82d523e32 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:44.478928 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83374e8-f24e-4512-a1ab-8cb82d523e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:44.479397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83374e8-f24e-4512-a1ab-8cb82d523e32 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:44.480482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83374e8-f24e-4512-a1ab-8cb82d523e32 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:44.480967 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83374e8-f24e-4512-a1ab-8cb82d523e32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:44.483167 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83374e8-f24e-4512-a1ab-8cb82d523e32 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:44.489716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 364/752] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:15:44.489716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c8798-5620-44fb-af29-cc23f283d470 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:44.529842 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 389/753] 10.222.0.39 () {56 vars in 1133 bytes} [Thu Jul 28 22:15:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:44.531075 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83374e8-f24e-4512-a1ab-8cb82d523e32 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:44.592320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368c8798-5620-44fb-af29-cc23f283d470 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:44.628842 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d7252-f497-4252-9d4c-fa16bdcccbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:44.629416 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d7252-f497-4252-9d4c-fa16bdcccbfc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:44.629808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d7252-f497-4252-9d4c-fa16bdcccbfc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:44.630144 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d7252-f497-4252-9d4c-fa16bdcccbfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:44.641677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d7252-f497-4252-9d4c-fa16bdcccbfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:44.643144 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d7252-f497-4252-9d4c-fa16bdcccbfc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:44.648559 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83374e8-f24e-4512-a1ab-8cb82d523e32 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:44.659632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c83d7252-f497-4252-9d4c-fa16bdcccbfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:44.659632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:44.659632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:44.659632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:44.659632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:44.659632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:44.659632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:44.659632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:44.659632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:44.659632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:44.659632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:44.659632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:44.659632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:44.659632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:44.659632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:44.660602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:44.660602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:44.660602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:44.660602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:44.660602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:44.660602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:44.660602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:44.660602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:44.660602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:44.660602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:44.660602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:44.660602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:44.660602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:44.660602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:44.660602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:44.660602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:44.660602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:44.662169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:44.662169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:44.662169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:44.662169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:44.662169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 365/754] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:44.662500 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d7252-f497-4252-9d4c-fa16bdcccbfc None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:44.682045 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:44.691659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fdc62-637a-46a0-a7ca-5f9691f21159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:44.692681 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fdc62-637a-46a0-a7ca-5f9691f21159 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:44.693199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fdc62-637a-46a0-a7ca-5f9691f21159 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:44.693739 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fdc62-637a-46a0-a7ca-5f9691f21159 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:44.694592 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fdc62-637a-46a0-a7ca-5f9691f21159 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:44.696303 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fdc62-637a-46a0-a7ca-5f9691f21159 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:44.706141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ba6fdc62-637a-46a0-a7ca-5f9691f21159 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 22:15:44.706141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:44.706141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:15:44.706141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:15:44.706141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:15:44.706141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:15:44.706141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:15:44.706141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:44.706141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 22:15:44.706141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:44.706141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:15:44.706141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:44.706141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:44.706141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:44.706141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:44.706141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:44.707131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:44.707131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:44.707131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:44.707131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:44.707131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:44.707131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:44.707131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:44.707131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:15:44.707131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:15:44.707131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:15:44.707131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:15:44.707131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:44.707131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:44.707131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:44.707131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:44.707131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:44.707993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:44.707993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:44.707993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:44.707993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:44.707993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:44.707993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:44.707993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:44.707993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:44.707993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:44.707993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:44.707993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:44.707993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:15:44.707993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:44.707993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 22:15:44.707993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:44.715720 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 390/755] 10.222.0.39 () {56 vars in 1129 bytes} [Thu Jul 28 22:15:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:44.716813 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fdc62-637a-46a0-a7ca-5f9691f21159 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:44.774709 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:44.782766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d7252-f497-4252-9d4c-fa16bdcccbfc None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:44.823215 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e56ab-0897-4955-b928-48de1810da45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:44.824204 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e56ab-0897-4955-b928-48de1810da45 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:44.824688 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e56ab-0897-4955-b928-48de1810da45 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:44.825166 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e56ab-0897-4955-b928-48de1810da45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:44.827656 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fdc62-637a-46a0-a7ca-5f9691f21159 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:44.838248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4e56ab-0897-4955-b928-48de1810da45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:44.840041 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4e56ab-0897-4955-b928-48de1810da45 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:44.856961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-8c4e56ab-0897-4955-b928-48de1810da45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:44.856961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:44.856961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:44.856961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:44.856961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:44.856961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:44.856961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:44.856961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:44.856961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:44.856961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:44.856961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:44.856961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:44.856961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:44.856961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:44.856961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:44.858139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:44.858139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:44.858139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:44.858139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:44.858139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:44.858139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:44.858139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:44.858139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:44.858139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:44.858139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:44.858139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:44.858139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:44.858139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:44.858139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:44.858139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:44.858139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:44.858139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:44.859239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:44.859239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:44.859239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:44.859239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:44.861205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 366/756] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:44.862291 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e56ab-0897-4955-b928-48de1810da45 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:44.863401 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29682cf-c940-4d96-ab99-9555a86a52bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:44.864115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29682cf-c940-4d96-ab99-9555a86a52bc None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:44.864493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29682cf-c940-4d96-ab99-9555a86a52bc None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:44.864860 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29682cf-c940-4d96-ab99-9555a86a52bc None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:44.865802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29682cf-c940-4d96-ab99-9555a86a52bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:44.866110 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29682cf-c940-4d96-ab99-9555a86a52bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:44.868381 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29682cf-c940-4d96-ab99-9555a86a52bc None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:44.885723 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 391/757] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:44.886649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29682cf-c940-4d96-ab99-9555a86a52bc None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:44.934921 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:44.982435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4e56ab-0897-4955-b928-48de1810da45 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:44.989823 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29682cf-c940-4d96-ab99-9555a86a52bc None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:45.004363 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:45.024703 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bd445-111b-43d7-838a-08bd10edc91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:45.025590 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bd445-111b-43d7-838a-08bd10edc91e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:45.026031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bd445-111b-43d7-838a-08bd10edc91e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:45.026485 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82bd445-111b-43d7-838a-08bd10edc91e None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:45.027335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82bd445-111b-43d7-838a-08bd10edc91e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:45.027587 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed8080-1493-41ed-835d-7626dbf35313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:45.027956 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82bd445-111b-43d7-838a-08bd10edc91e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:45.028385 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed8080-1493-41ed-835d-7626dbf35313 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:45.028757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed8080-1493-41ed-835d-7626dbf35313 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:45.029121 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed8080-1493-41ed-835d-7626dbf35313 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:45.029962 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed8080-1493-41ed-835d-7626dbf35313 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:45.030263 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82bd445-111b-43d7-838a-08bd10edc91e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:45.030435 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed8080-1493-41ed-835d-7626dbf35313 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:45.031882 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed8080-1493-41ed-835d-7626dbf35313 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:45.036905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 367/758] 10.222.0.39 () {56 vars in 1124 bytes} [Thu Jul 28 22:15:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:45.038095 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bd445-111b-43d7-838a-08bd10edc91e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:45.046240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 392/759] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:45.047194 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed8080-1493-41ed-835d-7626dbf35313 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:45.164185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed8080-1493-41ed-835d-7626dbf35313 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:45.183929 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82bd445-111b-43d7-838a-08bd10edc91e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:45.205828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a2f2a-5d8e-43e5-967b-85edea715123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:45.206457 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a2f2a-5d8e-43e5-967b-85edea715123 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:45.206744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a2f2a-5d8e-43e5-967b-85edea715123 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:45.207021 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a2f2a-5d8e-43e5-967b-85edea715123 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:45.208246 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a2f2a-5d8e-43e5-967b-85edea715123 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:45.208556 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a2f2a-5d8e-43e5-967b-85edea715123 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:45.210217 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0a2f2a-5d8e-43e5-967b-85edea715123 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:45.215016 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:45.222526 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 393/760] 10.222.0.39 () {56 vars in 1124 bytes} [Thu Jul 28 22:15:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:45.223373 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a2f2a-5d8e-43e5-967b-85edea715123 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:45.225884 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3e3e4-e68f-400c-ae8b-e64bf7edd0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:45.226749 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3e3e4-e68f-400c-ae8b-e64bf7edd0da None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:45.227519 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3e3e4-e68f-400c-ae8b-e64bf7edd0da None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:45.227994 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b3e3e4-e68f-400c-ae8b-e64bf7edd0da None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:45.236381 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3e3e4-e68f-400c-ae8b-e64bf7edd0da None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:45.238249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b3e3e4-e68f-400c-ae8b-e64bf7edd0da None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:45.250737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-33b3e3e4-e68f-400c-ae8b-e64bf7edd0da None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 22:15:45.250737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:45.250737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:45.250737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:45.250737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:45.250737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:45.250737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:45.250737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:45.250737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:45.250737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:45.250737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:45.250737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:45.250737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:45.250737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:45.250737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:45.250737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:45.251677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:45.251677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:45.251677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:45.251677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:45.251677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:45.251677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:45.251677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:45.251677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:45.251677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:45.251677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:45.251677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:45.251677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:45.251677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:45.251677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:45.251677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:45.251677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:45.251677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:45.251677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:45.252600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:45.252600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:45.252600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:45.252600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:45.252600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:45.252600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:45.252600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:45.252600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:45.252600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 22:15:45.252600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:45.260111 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 368/761] 10.222.0.39 () {56 vars in 1117 bytes} [Thu Jul 28 22:15:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:45.261258 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3e3e4-e68f-400c-ae8b-e64bf7edd0da None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:45.264555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:45.360035 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0a2f2a-5d8e-43e5-967b-85edea715123 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:45.369399 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b3e3e4-e68f-400c-ae8b-e64bf7edd0da None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:45.404839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbe374-4a1b-4a55-9b91-8046305c147a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:45.405825 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbe374-4a1b-4a55-9b91-8046305c147a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:45.406157 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacf3b8-a6e9-4f72-a224-0603e75872ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:45.406536 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbe374-4a1b-4a55-9b91-8046305c147a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:45.406860 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacf3b8-a6e9-4f72-a224-0603e75872ec None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:45.407144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacf3b8-a6e9-4f72-a224-0603e75872ec None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:45.407422 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bacf3b8-a6e9-4f72-a224-0603e75872ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:45.407631 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dbe374-4a1b-4a55-9b91-8046305c147a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:45.408570 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dbe374-4a1b-4a55-9b91-8046305c147a None admin] RBAC: Adding query filter params (interface=public, service_id=74f0be987dbe4b698ad6f98c14660542, region_id=RegionOne) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:45.409073 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dbe374-4a1b-4a55-9b91-8046305c147a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:45.410663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dbe374-4a1b-4a55-9b91-8046305c147a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:45.415260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacf3b8-a6e9-4f72-a224-0603e75872ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:45.417271 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bacf3b8-a6e9-4f72-a224-0603e75872ec None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:45.420170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 369/762] 10.222.0.39 () {56 vars in 1251 bytes} [Thu Jul 28 22:15:45 2022] GET /identity/v3/endpoints?service_id=74f0be987dbe4b698ad6f98c14660542&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:45.421258 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbe374-4a1b-4a55-9b91-8046305c147a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:45.431349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-1bacf3b8-a6e9-4f72-a224-0603e75872ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:45.431349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:45.431349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:45.431349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:45.431349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:45.431349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:45.431349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:45.431349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:45.431349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:45.431349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:45.431349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:45.431349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:45.431349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:45.431349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:45.431349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:45.432281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:45.432281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:45.432281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:45.432281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:45.432281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:45.432281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:45.432281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:45.432281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:45.432281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:45.432281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:45.432281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:45.432281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:45.432281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:45.432281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:45.432281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:45.432281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:45.432281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:45.433171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:45.433171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:45.433171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:45.433171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:45.433171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:45.433171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:45.433171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:45.433171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:45.435579 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 394/763] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:45.436683 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacf3b8-a6e9-4f72-a224-0603e75872ec None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:45.480600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:45.538222 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dbe374-4a1b-4a55-9b91-8046305c147a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:45.564196 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:45.571873 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bacf3b8-a6e9-4f72-a224-0603e75872ec None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:45.581744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65075b-8c32-4b3a-8bb2-4f5b04639df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:45.582373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65075b-8c32-4b3a-8bb2-4f5b04639df9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:45.582659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65075b-8c32-4b3a-8bb2-4f5b04639df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:45.582952 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65075b-8c32-4b3a-8bb2-4f5b04639df9 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:45.584110 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65075b-8c32-4b3a-8bb2-4f5b04639df9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:45.584421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65075b-8c32-4b3a-8bb2-4f5b04639df9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:45.586748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65075b-8c32-4b3a-8bb2-4f5b04639df9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:45.609686 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 370/764] 10.222.0.39 () {56 vars in 1149 bytes} [Thu Jul 28 22:15:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:45.610541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65075b-8c32-4b3a-8bb2-4f5b04639df9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:45.611652 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d480f2d-6dd0-4dd4-ba92-1f4f808ee84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:45.612520 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d480f2d-6dd0-4dd4-ba92-1f4f808ee84c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:45.613135 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d480f2d-6dd0-4dd4-ba92-1f4f808ee84c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:45.613730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d480f2d-6dd0-4dd4-ba92-1f4f808ee84c None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:45.614722 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d480f2d-6dd0-4dd4-ba92-1f4f808ee84c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:45.616414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d480f2d-6dd0-4dd4-ba92-1f4f808ee84c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:45.628033 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 395/765] 10.222.0.39 () {56 vars in 1087 bytes} [Thu Jul 28 22:15:45 2022] GET /identity/v3/services => generated 772 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:45.628033 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d480f2d-6dd0-4dd4-ba92-1f4f808ee84c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:45.746749 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65075b-8c32-4b3a-8bb2-4f5b04639df9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:45.748170 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d480f2d-6dd0-4dd4-ba92-1f4f808ee84c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:45.757919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:45.793785 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c6fd8-a774-4b9c-b5cf-5207ff820399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:45.794692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c6fd8-a774-4b9c-b5cf-5207ff820399 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:45.794979 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c6fd8-a774-4b9c-b5cf-5207ff820399 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:45.795258 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c6fd8-a774-4b9c-b5cf-5207ff820399 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:45.796020 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c6fd8-a774-4b9c-b5cf-5207ff820399 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:45.796333 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c6fd8-a774-4b9c-b5cf-5207ff820399 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:45.797562 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027c533-c19a-4af8-8144-af9863ccb69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:45.798007 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c6fd8-a774-4b9c-b5cf-5207ff820399 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:45.798200 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027c533-c19a-4af8-8144-af9863ccb69b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:45.798486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027c533-c19a-4af8-8144-af9863ccb69b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:45.798764 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027c533-c19a-4af8-8144-af9863ccb69b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:45.812428 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 371/766] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:45.813829 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6a1b2-5dac-451a-a9b2-4e2b295b07bd None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:45.814345 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6a1b2-5dac-451a-a9b2-4e2b295b07bd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:45.814345 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a6a1b2-5dac-451a-a9b2-4e2b295b07bd None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:45.815831 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 372/767] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:45.821448 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6a1b2-5dac-451a-a9b2-4e2b295b07bd None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:45.839645 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:45.901898 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027c533-c19a-4af8-8144-af9863ccb69b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:45.903399 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5027c533-c19a-4af8-8144-af9863ccb69b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:45.934728 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a6a1b2-5dac-451a-a9b2-4e2b295b07bd None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:45.971674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903eb176-99d3-42ca-9300-c5f2bbb80482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:45.972394 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903eb176-99d3-42ca-9300-c5f2bbb80482 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:45.972684 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903eb176-99d3-42ca-9300-c5f2bbb80482 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:45.972961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903eb176-99d3-42ca-9300-c5f2bbb80482 None admin] PATH_INFO: `/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/77efe8046ea0468cb0c7c58c24fc4e75/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:45.974974 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903eb176-99d3-42ca-9300-c5f2bbb80482 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user_id=77efe8046ea0468cb0c7c58c24fc4e75, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:45.976525 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903eb176-99d3-42ca-9300-c5f2bbb80482 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:45.995255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 373/768] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:15:45 2022] PUT /identity/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/77efe8046ea0468cb0c7c58c24fc4e75/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:15:45.996583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a0b89-589d-4b12-9be1-8ba26494e630 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:45.996930 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a0b89-589d-4b12-9be1-8ba26494e630 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:45.997424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980a0b89-589d-4b12-9be1-8ba26494e630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:46.025016 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-980a0b89-589d-4b12-9be1-8ba26494e630 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:46.026862 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-980a0b89-589d-4b12-9be1-8ba26494e630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:46.048477 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 396/769] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 22:15:46.058634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027c533-c19a-4af8-8144-af9863ccb69b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:46.062437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:46.175525 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 374/770] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:15:46.203171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:46.216427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a0b89-589d-4b12-9be1-8ba26494e630 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:46.242066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027c533-c19a-4af8-8144-af9863ccb69b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:46.279848 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77894f7d-93d6-4bde-851a-ab7be145e42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:46.280865 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77894f7d-93d6-4bde-851a-ab7be145e42a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:46.281371 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77894f7d-93d6-4bde-851a-ab7be145e42a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:46.281886 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77894f7d-93d6-4bde-851a-ab7be145e42a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:46.295374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77894f7d-93d6-4bde-851a-ab7be145e42a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:46.297139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77894f7d-93d6-4bde-851a-ab7be145e42a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:46.320131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-77894f7d-93d6-4bde-851a-ab7be145e42a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:46.320131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:46.320131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:46.320131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:46.320131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:46.320131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:46.320131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:46.320131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:46.320131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:46.320131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:46.320131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:46.320131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:46.320131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:46.320131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:46.320131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:46.322282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:46.322282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:46.322282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:46.322282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:46.322282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:46.322282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:46.322282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:46.322282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:46.322282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:46.322282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:46.322282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:46.322282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:46.322282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:46.322282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:46.322282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:46.322282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:46.322282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:46.323199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:46.323199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:46.323199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:46.323199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:46.325433 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 397/771] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:46.330729 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77894f7d-93d6-4bde-851a-ab7be145e42a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:46.337060 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980a0b89-589d-4b12-9be1-8ba26494e630 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:46.392879 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309b847-4c60-4786-925a-41950ba8a71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:46.393556 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309b847-4c60-4786-925a-41950ba8a71e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:46.393843 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309b847-4c60-4786-925a-41950ba8a71e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:46.394125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c309b847-4c60-4786-925a-41950ba8a71e None admin] PATH_INFO: `/v3/services/network` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:46.394764 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c309b847-4c60-4786-925a-41950ba8a71e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:46.396147 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c309b847-4c60-4786-925a-41950ba8a71e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:46.420794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c309b847-4c60-4786-925a-41950ba8a71e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 22:15:46.420794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:46.420794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:15:46.420794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:15:46.420794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:15:46.420794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:15:46.420794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:15:46.420794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:46.420794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 22:15:46.420794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:46.420794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:15:46.420794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:46.420794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:46.420794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:46.420794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:46.420794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:46.420794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:46.421883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:46.421883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:46.421883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:46.421883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:46.421883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:46.421883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:46.421883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:15:46.421883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:15:46.421883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:15:46.421883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:15:46.421883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:46.421883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:46.421883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:46.421883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:46.421883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:46.421883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:46.422720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:46.422720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:46.422720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:46.422720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:46.422720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:46.422720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:46.422720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:46.422720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:46.422720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:46.422720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:46.422720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:15:46.422720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:46.422720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 22:15:46.422720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:46.422720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 375/772] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:46 2022] GET /identity/v3/services/network => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:15:46.430518 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309b847-4c60-4786-925a-41950ba8a71e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:46.471866 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:46.499469 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77894f7d-93d6-4bde-851a-ab7be145e42a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:46.514198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:46.539107 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025fa662-77a9-4c33-8c10-7b51857864dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:46.539909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fa662-77a9-4c33-8c10-7b51857864dd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:46.540250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fa662-77a9-4c33-8c10-7b51857864dd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:46.540641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025fa662-77a9-4c33-8c10-7b51857864dd None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:46.541351 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025fa662-77a9-4c33-8c10-7b51857864dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:46.541746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025fa662-77a9-4c33-8c10-7b51857864dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:46.543312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025fa662-77a9-4c33-8c10-7b51857864dd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:46.562827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 398/773] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:46.562827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01f79d-5b7b-4d02-9016-0efc8c91e6f5 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:46.562827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01f79d-5b7b-4d02-9016-0efc8c91e6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:46.562827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b01f79d-5b7b-4d02-9016-0efc8c91e6f5 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:46.562827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 399/774] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:46.562827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01f79d-5b7b-4d02-9016-0efc8c91e6f5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:46.579006 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c309b847-4c60-4786-925a-41950ba8a71e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:46.631442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1455f-6fac-4cd4-b506-506af8a10623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:46.633359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1455f-6fac-4cd4-b506-506af8a10623 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:46.634182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1455f-6fac-4cd4-b506-506af8a10623 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:46.634601 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1455f-6fac-4cd4-b506-506af8a10623 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:46.636056 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1455f-6fac-4cd4-b506-506af8a10623 None admin] RBAC: Adding query filter params (name=network) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:46.636509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1455f-6fac-4cd4-b506-506af8a10623 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:46.638437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1455f-6fac-4cd4-b506-506af8a10623 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:46.645802 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 376/775] 10.222.0.39 () {56 vars in 1112 bytes} [Thu Jul 28 22:15:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:46.647494 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5608031-fba2-48cd-a36a-9e60ec06c0b8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:46.648180 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5608031-fba2-48cd-a36a-9e60ec06c0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:46.650090 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5608031-fba2-48cd-a36a-9e60ec06c0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:46.673369 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b5608031-fba2-48cd-a36a-9e60ec06c0b8 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:46.675070 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b5608031-fba2-48cd-a36a-9e60ec06c0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:46.684599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b01f79d-5b7b-4d02-9016-0efc8c91e6f5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:46.726735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051cbdf-a4c3-4f60-8148-1d81a909657a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:46.728219 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051cbdf-a4c3-4f60-8148-1d81a909657a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:46.728508 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051cbdf-a4c3-4f60-8148-1d81a909657a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:46.728785 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051cbdf-a4c3-4f60-8148-1d81a909657a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:46.735916 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051cbdf-a4c3-4f60-8148-1d81a909657a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:46.737381 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5051cbdf-a4c3-4f60-8148-1d81a909657a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:46.749631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-5051cbdf-a4c3-4f60-8148-1d81a909657a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:46.749631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:46.749631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:46.749631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:46.749631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:46.749631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:46.749631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:46.749631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:46.749631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:46.749631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:46.749631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:46.749631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:46.749631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:46.749631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:46.749631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:46.750554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:46.750554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:46.750554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:46.750554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:46.750554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:46.750554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:46.750554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:46.750554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:46.750554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:46.750554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:46.750554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:46.750554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:46.750554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:46.750554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:46.750554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:46.750554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:46.750554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:46.751441 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:46.751441 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:46.751441 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:46.751441 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:46.751441 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:46.751441 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:46.751441 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:46.751441 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:46.755667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 400/776] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:46.756509 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051cbdf-a4c3-4f60-8148-1d81a909657a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:46.763113 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 377/777] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:15:46.764435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5608031-fba2-48cd-a36a-9e60ec06c0b8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:46.790725 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:46.792313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:46.868488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051cbdf-a4c3-4f60-8148-1d81a909657a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:46.907160 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36408593-7f78-495f-b258-b90184d6ac42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:46.908203 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36408593-7f78-495f-b258-b90184d6ac42 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:46.908393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36408593-7f78-495f-b258-b90184d6ac42 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:46.908767 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36408593-7f78-495f-b258-b90184d6ac42 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:46.909574 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36408593-7f78-495f-b258-b90184d6ac42 None admin] RBAC: Adding query filter params (type=network) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:46.910005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36408593-7f78-495f-b258-b90184d6ac42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:46.911575 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36408593-7f78-495f-b258-b90184d6ac42 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:46.916152 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5608031-fba2-48cd-a36a-9e60ec06c0b8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:46.917466 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 401/778] 10.222.0.39 () {56 vars in 1112 bytes} [Thu Jul 28 22:15:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:46.918563 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36408593-7f78-495f-b258-b90184d6ac42 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:46.963945 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a9868-a4d7-4247-b239-466bd3959884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:46.965166 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a9868-a4d7-4247-b239-466bd3959884 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:46.965166 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a9868-a4d7-4247-b239-466bd3959884 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:46.965628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a9868-a4d7-4247-b239-466bd3959884 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:46.966591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a9868-a4d7-4247-b239-466bd3959884 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:46.967028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a9868-a4d7-4247-b239-466bd3959884 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:46.968676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a9868-a4d7-4247-b239-466bd3959884 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:46.987173 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 378/779] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:46.988181 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a9868-a4d7-4247-b239-466bd3959884 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:47.034322 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:47.044304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36408593-7f78-495f-b258-b90184d6ac42 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:47.082825 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:47.086379 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061f5e9-26ef-4ec8-b98f-22b44333d167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:47.087501 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061f5e9-26ef-4ec8-b98f-22b44333d167 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:47.087992 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061f5e9-26ef-4ec8-b98f-22b44333d167 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:47.088473 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2061f5e9-26ef-4ec8-b98f-22b44333d167 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:47.100684 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2061f5e9-26ef-4ec8-b98f-22b44333d167 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:47.106950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-2061f5e9-26ef-4ec8-b98f-22b44333d167 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:15:47.106950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:47.106950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:47.106950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:47.106950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:47.106950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:47.106950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:47.106950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:47.106950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:47.106950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:47.106950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:47.106950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:47.106950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:47.106950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:47.106950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:47.106950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:47.107978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:47.107978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:47.107978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:47.107978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:47.107978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:47.107978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:47.107978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:47.107978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:47.107978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:47.107978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:47.107978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:47.107978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:47.107978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:47.107978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:47.107978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:47.107978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:47.108849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:47.108849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:47.108849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:47.108849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:47.108849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:47.108849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:47.108849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:47.108849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:47.108849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:47.108849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:47.108849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:15:47.108849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:47.109501 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a9868-a4d7-4247-b239-466bd3959884 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:47.111808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 402/780] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:15:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:47.112905 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061f5e9-26ef-4ec8-b98f-22b44333d167 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:47.155520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec0102-b3fb-417b-b846-aea70bd5e804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:47.156157 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec0102-b3fb-417b-b846-aea70bd5e804 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:47.156487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec0102-b3fb-417b-b846-aea70bd5e804 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:47.156770 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec0102-b3fb-417b-b846-aea70bd5e804 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:47.157618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec0102-b3fb-417b-b846-aea70bd5e804 None admin] RBAC: Adding query filter params (interface=public, service_id=9d48195d48674b64ae873356c0e132f4, region_id=RegionOne) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:47.157895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec0102-b3fb-417b-b846-aea70bd5e804 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:47.159398 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec0102-b3fb-417b-b846-aea70bd5e804 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:47.165997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 379/781] 10.222.0.39 () {56 vars in 1251 bytes} [Thu Jul 28 22:15:46 2022] GET /identity/v3/endpoints?service_id=9d48195d48674b64ae873356c0e132f4&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:47.167040 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec0102-b3fb-417b-b846-aea70bd5e804 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:47.225365 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2061f5e9-26ef-4ec8-b98f-22b44333d167 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:47.263325 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e5053-b672-49f4-94af-92407662e947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:47.264550 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e5053-b672-49f4-94af-92407662e947 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:47.265040 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e5053-b672-49f4-94af-92407662e947 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:47.265520 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e5053-b672-49f4-94af-92407662e947 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:47.266813 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e5053-b672-49f4-94af-92407662e947 None admin] RBAC: Adding query filter params (role.id=a66ab77a63174592a1e468d2464e8bc4, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=85626350279c40ea99da93b9ed6994b2) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:47.267334 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e5053-b672-49f4-94af-92407662e947 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:47.269071 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e5053-b672-49f4-94af-92407662e947 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:47.281702 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec0102-b3fb-417b-b846-aea70bd5e804 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:47.282108 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 403/782] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:15:47 2022] GET /identity/v3/role_assignments?user.id=85626350279c40ea99da93b9ed6994b2&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=a66ab77a63174592a1e468d2464e8bc4 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:47.283426 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e5053-b672-49f4-94af-92407662e947 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:47.318558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac6060-0106-432e-878a-d5b8888462ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:47.319164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac6060-0106-432e-878a-d5b8888462ab None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:47.319479 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac6060-0106-432e-878a-d5b8888462ab None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:47.320079 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ac6060-0106-432e-878a-d5b8888462ab None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:47.320905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac6060-0106-432e-878a-d5b8888462ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:47.321165 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac6060-0106-432e-878a-d5b8888462ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:47.322506 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ac6060-0106-432e-878a-d5b8888462ab None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:47.331116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:47.333996 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 380/783] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:15:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:47.344161 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac6060-0106-432e-878a-d5b8888462ab None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:47.344902 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:47.459643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e5053-b672-49f4-94af-92407662e947 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:47.476652 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ac6060-0106-432e-878a-d5b8888462ab None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:47.500425 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a32b5d-6cf3-42b3-b80f-db7dd59800a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:47.501439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a32b5d-6cf3-42b3-b80f-db7dd59800a1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:47.501996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a32b5d-6cf3-42b3-b80f-db7dd59800a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:47.502592 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a32b5d-6cf3-42b3-b80f-db7dd59800a1 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:47.503597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a32b5d-6cf3-42b3-b80f-db7dd59800a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:47.505250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a32b5d-6cf3-42b3-b80f-db7dd59800a1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:47.512000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 404/784] 10.222.0.39 () {56 vars in 1087 bytes} [Thu Jul 28 22:15:47 2022] GET /identity/v3/services => generated 772 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:47.521654 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f099f-9454-435d-bcc6-7787485ab100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:47.522330 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f099f-9454-435d-bcc6-7787485ab100 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:47.522618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f099f-9454-435d-bcc6-7787485ab100 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:47.522901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f099f-9454-435d-bcc6-7787485ab100 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:47.544862 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f099f-9454-435d-bcc6-7787485ab100 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:47.546466 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f099f-9454-435d-bcc6-7787485ab100 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:47.570363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-339f099f-9454-435d-bcc6-7787485ab100 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:47.570363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:47.570363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:47.570363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:47.570363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:47.570363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:47.570363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:47.570363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:47.570363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:47.570363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:47.570363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:47.570363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:47.570363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:47.570363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:47.570363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:47.571917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:47.571917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:47.571917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:47.571917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:47.571917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:47.571917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:47.571917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:47.571917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:47.571917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:47.571917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:47.571917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:47.571917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:47.571917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:47.571917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:47.571917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:47.571917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:47.571917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:47.572831 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:47.572831 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:47.572831 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:47.572831 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:47.575271 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 381/785] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:15:47.580935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a32b5d-6cf3-42b3-b80f-db7dd59800a1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:47.685236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:47.705140 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a32b5d-6cf3-42b3-b80f-db7dd59800a1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:47.755279 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a36b23-be60-4ce8-8329-85eab39ae7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:47.755908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a36b23-be60-4ce8-8329-85eab39ae7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:47.756194 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a36b23-be60-4ce8-8329-85eab39ae7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:47.756472 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a36b23-be60-4ce8-8329-85eab39ae7d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:47.757174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a36b23-be60-4ce8-8329-85eab39ae7d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:47.757486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a36b23-be60-4ce8-8329-85eab39ae7d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:47.759126 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a36b23-be60-4ce8-8329-85eab39ae7d3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:47.779644 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 405/786] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:47.781451 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f099f-9454-435d-bcc6-7787485ab100 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:47.847060 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:47.895853 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f099f-9454-435d-bcc6-7787485ab100 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:47.934451 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78621f4-1cb5-4d83-9ab8-7877a8c4fa28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:47.935099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78621f4-1cb5-4d83-9ab8-7877a8c4fa28 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:47.935381 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78621f4-1cb5-4d83-9ab8-7877a8c4fa28 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:47.935654 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78621f4-1cb5-4d83-9ab8-7877a8c4fa28 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:47.943251 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcae87-1519-4157-87b8-44c05697476c None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:47.943733 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcae87-1519-4157-87b8-44c05697476c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:47.943836 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcae87-1519-4157-87b8-44c05697476c None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:47.944083 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78621f4-1cb5-4d83-9ab8-7877a8c4fa28 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:47.945381 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 406/787] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:15:47.946311 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78621f4-1cb5-4d83-9ab8-7877a8c4fa28 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:47.951254 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564d669-8fda-4667-937f-fc473297929b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:47.951542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564d669-8fda-4667-937f-fc473297929b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:47.951814 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564d669-8fda-4667-937f-fc473297929b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:47.964322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a78621f4-1cb5-4d83-9ab8-7877a8c4fa28 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 22:15:47.964322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:47.964322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:47.964322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:47.964322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:47.964322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:47.964322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:47.964322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:47.964322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:47.964322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:47.964322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:47.964322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:47.964322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:47.964322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:47.964322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:47.964322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:47.965315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:47.965315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:47.965315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:47.965315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:47.965315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:47.965315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:47.965315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:47.965315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:47.965315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:47.965315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:47.965315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:47.965315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:47.965315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:47.965315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:47.965315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:47.965315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:47.965315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:47.965315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:47.966316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:47.966316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:47.966316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:47.966316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:47.966316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:47.966316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:47.966316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:47.966316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:47.966316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 22:15:47.966316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:47.966316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 382/788] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:15:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:47.971137 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5564d669-8fda-4667-937f-fc473297929b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:47.971527 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78621f4-1cb5-4d83-9ab8-7877a8c4fa28 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:47.972321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5564d669-8fda-4667-937f-fc473297929b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:48.084603 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 407/789] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:15:48.112453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564d669-8fda-4667-937f-fc473297929b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:48.116703 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78621f4-1cb5-4d83-9ab8-7877a8c4fa28 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:48.132066 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:48.169849 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40615d9f-d3e2-4412-887c-1a779d754bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:48.170583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40615d9f-d3e2-4412-887c-1a779d754bd8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:48.170871 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40615d9f-d3e2-4412-887c-1a779d754bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:48.176805 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40615d9f-d3e2-4412-887c-1a779d754bd8 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:48.178078 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40615d9f-d3e2-4412-887c-1a779d754bd8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:48.178393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40615d9f-d3e2-4412-887c-1a779d754bd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:48.179887 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40615d9f-d3e2-4412-887c-1a779d754bd8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:48.214755 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:48.216715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 383/790] 10.222.0.39 () {56 vars in 1135 bytes} [Thu Jul 28 22:15:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:48.223412 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40615d9f-d3e2-4412-887c-1a779d754bd8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:48.265072 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564d669-8fda-4667-937f-fc473297929b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:48.322264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae6b67-3371-49bf-aa57-3dd809b1e83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:48.322935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae6b67-3371-49bf-aa57-3dd809b1e83b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:48.323224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae6b67-3371-49bf-aa57-3dd809b1e83b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:48.323504 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae6b67-3371-49bf-aa57-3dd809b1e83b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:48.324152 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae6b67-3371-49bf-aa57-3dd809b1e83b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:48.325551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae6b67-3371-49bf-aa57-3dd809b1e83b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:48.347966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-51ae6b67-3371-49bf-aa57-3dd809b1e83b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 22:15:48.347966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:48.347966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:15:48.347966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:15:48.347966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:15:48.347966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:15:48.347966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:15:48.347966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:48.347966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 22:15:48.347966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:48.347966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:15:48.347966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:48.347966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:48.347966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:48.347966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:48.347966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:48.349060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:48.349060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:48.349060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:48.349060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:48.349060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:48.349060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:48.349060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:48.349060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:15:48.349060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:15:48.349060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:15:48.349060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:15:48.349060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:48.349060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:48.349060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:48.349060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:48.349060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:48.349940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:48.349940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:48.349940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:48.349940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:48.349940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:48.349940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:48.349940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:48.349940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:48.349940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:48.349940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:48.349940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:48.349940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:15:48.349940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:48.349940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 28 22:15:48.349940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:48.350727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 408/791] 10.222.0.39 () {56 vars in 1129 bytes} [Thu Jul 28 22:15:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:15:48.355649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae6b67-3371-49bf-aa57-3dd809b1e83b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:48.358470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40615d9f-d3e2-4412-887c-1a779d754bd8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:48.395176 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21235d64-90ec-4ebb-a9bb-932492b2e0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:48.395742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21235d64-90ec-4ebb-a9bb-932492b2e0cf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:48.396074 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21235d64-90ec-4ebb-a9bb-932492b2e0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:48.396361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21235d64-90ec-4ebb-a9bb-932492b2e0cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:48.407915 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21235d64-90ec-4ebb-a9bb-932492b2e0cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:48.409145 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21235d64-90ec-4ebb-a9bb-932492b2e0cf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:48.424512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-21235d64-90ec-4ebb-a9bb-932492b2e0cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:48.424512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:48.424512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:48.424512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:48.424512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:48.424512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:48.424512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:48.424512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:48.424512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:48.424512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:48.424512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:48.424512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:48.424512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:48.424512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:48.424512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:48.426715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:48.426715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:48.426715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:48.426715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:48.426715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:48.426715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:48.426715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:48.426715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:48.426715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:48.426715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:48.426715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:48.426715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:48.426715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:48.426715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:48.426715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:48.426715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:48.426715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:48.428962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:48.428962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:48.428962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:48.428962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:48.428962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 384/792] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:15:48.428962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c87b9-ed45-4a55-badc-bd603eb39fef None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:48.428962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c87b9-ed45-4a55-badc-bd603eb39fef None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:48.428962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c87b9-ed45-4a55-badc-bd603eb39fef None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:48.430177 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 385/793] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:48.434110 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c87b9-ed45-4a55-badc-bd603eb39fef None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:48.474436 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:48.491036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae6b67-3371-49bf-aa57-3dd809b1e83b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:48.529212 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ae0ec-76c2-4f1f-80a6-f70caff54d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:48.529856 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ae0ec-76c2-4f1f-80a6-f70caff54d04 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:48.530143 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ae0ec-76c2-4f1f-80a6-f70caff54d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:48.530423 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ae0ec-76c2-4f1f-80a6-f70caff54d04 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:48.531271 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ae0ec-76c2-4f1f-80a6-f70caff54d04 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:48.531586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ae0ec-76c2-4f1f-80a6-f70caff54d04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:48.533173 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ae0ec-76c2-4f1f-80a6-f70caff54d04 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:48.538441 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 409/794] 10.222.0.39 () {56 vars in 1124 bytes} [Thu Jul 28 22:15:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:48.539768 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133321fe-7189-40a9-8ea4-4716a56f33e8 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:48.540057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133321fe-7189-40a9-8ea4-4716a56f33e8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:48.540337 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133321fe-7189-40a9-8ea4-4716a56f33e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:48.547686 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:48.561395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-133321fe-7189-40a9-8ea4-4716a56f33e8 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:48.562796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-133321fe-7189-40a9-8ea4-4716a56f33e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:48.576863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c87b9-ed45-4a55-badc-bd603eb39fef None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:48.629475 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae04cd3-3cc1-4a85-b8a0-b6b584742740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:48.630382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae04cd3-3cc1-4a85-b8a0-b6b584742740 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:48.630732 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae04cd3-3cc1-4a85-b8a0-b6b584742740 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:48.631011 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae04cd3-3cc1-4a85-b8a0-b6b584742740 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:48.632015 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae04cd3-3cc1-4a85-b8a0-b6b584742740 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:48.632726 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae04cd3-3cc1-4a85-b8a0-b6b584742740 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:48.635063 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae04cd3-3cc1-4a85-b8a0-b6b584742740 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:48.640961 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 410/795] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:15:48.642041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133321fe-7189-40a9-8ea4-4716a56f33e8 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:48.651836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 386/796] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:48.659528 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae04cd3-3cc1-4a85-b8a0-b6b584742740 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:48.749633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:48.753618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133321fe-7189-40a9-8ea4-4716a56f33e8 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:48.767405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae04cd3-3cc1-4a85-b8a0-b6b584742740 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:48.786929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:48.793876 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3f22d-662b-451f-8125-ace1f712a16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:48.794506 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3f22d-662b-451f-8125-ace1f712a16c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:48.794835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3f22d-662b-451f-8125-ace1f712a16c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:48.795120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3f22d-662b-451f-8125-ace1f712a16c None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:48.795812 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3f22d-662b-451f-8125-ace1f712a16c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:48.796121 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3f22d-662b-451f-8125-ace1f712a16c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:48.797500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3f22d-662b-451f-8125-ace1f712a16c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:48.803117 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 411/797] 10.222.0.39 () {56 vars in 1124 bytes} [Thu Jul 28 22:15:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:48.803982 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3f22d-662b-451f-8125-ace1f712a16c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:48.810879 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b2954c-4ea1-4d06-b631-75d3a6126ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:48.811522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2954c-4ea1-4d06-b631-75d3a6126ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:48.811801 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2954c-4ea1-4d06-b631-75d3a6126ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:48.812071 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b2954c-4ea1-4d06-b631-75d3a6126ad3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:48.819361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b2954c-4ea1-4d06-b631-75d3a6126ad3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:48.821451 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b2954c-4ea1-4d06-b631-75d3a6126ad3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:48.833080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a6b2954c-4ea1-4d06-b631-75d3a6126ad3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:48.833080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:48.833080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:48.833080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:48.833080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:48.833080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:48.833080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:48.833080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:48.833080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:48.833080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:48.833080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:48.833080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:48.833080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:48.833080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:48.833080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:48.835029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:48.835029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:48.835029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:48.835029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:48.835029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:48.835029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:48.835029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:48.835029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:48.835029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:48.835029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:48.835029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:48.835029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:48.835029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:48.835029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:48.835029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:48.835029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:48.835029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:48.837777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:48.837777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:48.837777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:48.837777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:48.837777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:48.837777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:48.837777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:48.837777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:48.837777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 387/798] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:48.837777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b2954c-4ea1-4d06-b631-75d3a6126ad3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:48.920189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3f22d-662b-451f-8125-ace1f712a16c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:48.943744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b2954c-4ea1-4d06-b631-75d3a6126ad3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:48.955911 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a6be7-0570-4280-897e-e6cb2c136133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:48.956632 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a6be7-0570-4280-897e-e6cb2c136133 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:48.956924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a6be7-0570-4280-897e-e6cb2c136133 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:48.957206 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5a6be7-0570-4280-897e-e6cb2c136133 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:48.968836 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5a6be7-0570-4280-897e-e6cb2c136133 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:48.975039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-5f5a6be7-0570-4280-897e-e6cb2c136133 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:15:48.975039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:48.975039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:48.975039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:48.975039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:48.975039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:48.975039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:48.975039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:48.975039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:48.975039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:48.975039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:48.975039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:48.975039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:48.975039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:48.975039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:48.975039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:48.977913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:48.977913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:48.977913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:48.977913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:48.977913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:48.977913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:48.977913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:48.977913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:48.977913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:48.977913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:48.977913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:48.977913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:48.977913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:48.977913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:48.977913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:48.977913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:48.979884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:48.979884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:48.979884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:48.979884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:48.979884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:48.979884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:48.979884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:48.979884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:48.979884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:48.979884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:48.979884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:15:48.979884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:48.979884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 412/799] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:15:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:48.979884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a6be7-0570-4280-897e-e6cb2c136133 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:48.988387 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e778d-0637-45d3-8d66-fff7cdbe7159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:48.988787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e778d-0637-45d3-8d66-fff7cdbe7159 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:48.989067 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e778d-0637-45d3-8d66-fff7cdbe7159 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:48.989338 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e778d-0637-45d3-8d66-fff7cdbe7159 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:48.990085 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e778d-0637-45d3-8d66-fff7cdbe7159 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:48.991502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e778d-0637-45d3-8d66-fff7cdbe7159 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:49.011885 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:49.027139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 388/800] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:15:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:15:49.028363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e778d-0637-45d3-8d66-fff7cdbe7159 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:49.043879 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:49.096859 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5a6be7-0570-4280-897e-e6cb2c136133 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:49.150778 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54648ea8-f121-40b4-bae4-eac474f0b8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:49.151702 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54648ea8-f121-40b4-bae4-eac474f0b8bc None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:49.152136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54648ea8-f121-40b4-bae4-eac474f0b8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:49.152558 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54648ea8-f121-40b4-bae4-eac474f0b8bc None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:49.155430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54648ea8-f121-40b4-bae4-eac474f0b8bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:49.155921 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54648ea8-f121-40b4-bae4-eac474f0b8bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:49.157647 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54648ea8-f121-40b4-bae4-eac474f0b8bc None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:49.177863 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 413/801] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:49.179481 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54648ea8-f121-40b4-bae4-eac474f0b8bc None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:49.250989 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e778d-0637-45d3-8d66-fff7cdbe7159 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:49.312110 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53e5a7-8f80-40c0-b75f-3b16803b9bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:49.312767 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54648ea8-f121-40b4-bae4-eac474f0b8bc None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:49.313008 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53e5a7-8f80-40c0-b75f-3b16803b9bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:49.313419 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53e5a7-8f80-40c0-b75f-3b16803b9bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:49.313864 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e53e5a7-8f80-40c0-b75f-3b16803b9bf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:49.315041 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53e5a7-8f80-40c0-b75f-3b16803b9bf7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:49.315403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53e5a7-8f80-40c0-b75f-3b16803b9bf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:49.317590 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e53e5a7-8f80-40c0-b75f-3b16803b9bf7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:49.329660 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:49.330510 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 389/802] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:15:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:49.336601 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53e5a7-8f80-40c0-b75f-3b16803b9bf7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:49.359384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf5213-30fb-40f4-b213-bdbfe67298a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:49.360372 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf5213-30fb-40f4-b213-bdbfe67298a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:49.360844 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf5213-30fb-40f4-b213-bdbfe67298a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:49.361305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf5213-30fb-40f4-b213-bdbfe67298a6 None admin] PATH_INFO: `/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/e3c9107872f147f6ab8698b67bb545af/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:49.364515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf5213-30fb-40f4-b213-bdbfe67298a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user_id=e3c9107872f147f6ab8698b67bb545af, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:49.367076 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf5213-30fb-40f4-b213-bdbfe67298a6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:49.393450 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 414/803] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:15:49 2022] PUT /identity/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/e3c9107872f147f6ab8698b67bb545af/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:15:49.426059 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:49.479327 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e53e5a7-8f80-40c0-b75f-3b16803b9bf7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:49.531487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2fe95e-6414-4dca-92da-c57c4545032d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:49.532207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fe95e-6414-4dca-92da-c57c4545032d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:49.532494 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fe95e-6414-4dca-92da-c57c4545032d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:49.532767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fe95e-6414-4dca-92da-c57c4545032d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:49.550178 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2fe95e-6414-4dca-92da-c57c4545032d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:49.551689 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2fe95e-6414-4dca-92da-c57c4545032d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:49.571274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-fd2fe95e-6414-4dca-92da-c57c4545032d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:49.571274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:49.571274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:49.571274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:49.571274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:49.571274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:49.571274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:49.571274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:49.571274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:49.571274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:49.571274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:49.571274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:49.571274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:49.571274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:49.571274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:49.572254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:49.572254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:49.572254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:49.572254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:49.572254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:49.572254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:49.572254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:49.572254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:49.572254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:49.572254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:49.572254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:49.572254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:49.572254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:49.572254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:49.572254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:49.572254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:49.572254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:49.573814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:49.573814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:49.573814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:49.573814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:49.573814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 390/804] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:15:49.578219 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf5213-30fb-40f4-b213-bdbfe67298a6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:49.695849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf5213-30fb-40f4-b213-bdbfe67298a6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:49.733894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de1834a-070a-4dfc-b533-882708c5f3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:49.734870 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1834a-070a-4dfc-b533-882708c5f3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:49.735358 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1834a-070a-4dfc-b533-882708c5f3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:49.735865 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de1834a-070a-4dfc-b533-882708c5f3f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:49.736769 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1834a-070a-4dfc-b533-882708c5f3f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:49.737280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1834a-070a-4dfc-b533-882708c5f3f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:49.738927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de1834a-070a-4dfc-b533-882708c5f3f5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:49.752644 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 415/805] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:49.757778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2fe95e-6414-4dca-92da-c57c4545032d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:49.934086 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2fe95e-6414-4dca-92da-c57c4545032d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:49.958649 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:49.982466 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d450c-9a7f-4fab-9ec0-3e55fa60d453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:49.983111 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d450c-9a7f-4fab-9ec0-3e55fa60d453 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:49.983393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d450c-9a7f-4fab-9ec0-3e55fa60d453 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:49.983670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d450c-9a7f-4fab-9ec0-3e55fa60d453 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:49.987834 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05047169-346e-47dd-8be6-703e486692f8 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:49.988293 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05047169-346e-47dd-8be6-703e486692f8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:49.988384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05047169-346e-47dd-8be6-703e486692f8 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:49.993557 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 416/806] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:49.994837 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74095f85-f124-4869-ada2-acd2904d54a0 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:49.995198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74095f85-f124-4869-ada2-acd2904d54a0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:49.995575 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74095f85-f124-4869-ada2-acd2904d54a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:50.002441 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d450c-9a7f-4fab-9ec0-3e55fa60d453 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:50.004136 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d450c-9a7f-4fab-9ec0-3e55fa60d453 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:50.020692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a94d450c-9a7f-4fab-9ec0-3e55fa60d453 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 22:15:50.020692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:50.020692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:50.020692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:50.020692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:50.020692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:50.020692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:50.020692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:50.020692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:50.020692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:50.020692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:50.020692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:50.020692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:50.020692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:50.020692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:50.020692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:50.021717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:50.021717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:50.021717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:50.021717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:50.021717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:50.021717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:50.021717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:50.021717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:50.021717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:50.021717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:50.021717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:50.021717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:50.021717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:50.021717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:50.021717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:50.021717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:50.021717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:50.021717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:50.022676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:50.022676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:50.022676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:50.022676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:50.022676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:50.022676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:50.022676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:50.022676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:50.022676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 22:15:50.022676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:50.023222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-74095f85-f124-4869-ada2-acd2904d54a0 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:50.024916 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-74095f85-f124-4869-ada2-acd2904d54a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:50.026370 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 391/807] 10.222.0.39 () {56 vars in 1093 bytes} [Thu Jul 28 22:15:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:50.027757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb7438-e0a9-4087-9d43-0c40c74ef0cf None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:50.028190 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb7438-e0a9-4087-9d43-0c40c74ef0cf None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:50.028600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deb7438-e0a9-4087-9d43-0c40c74ef0cf None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:50.030055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 392/808] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:50.031666 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb7438-e0a9-4087-9d43-0c40c74ef0cf None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:50.065237 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:50.116901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 417/809] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:15:50.119671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2576a-e101-4373-b6ff-8543eca5dd53 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:50.120175 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2576a-e101-4373-b6ff-8543eca5dd53 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:50.120655 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd2576a-e101-4373-b6ff-8543eca5dd53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:50.151609 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3cd2576a-e101-4373-b6ff-8543eca5dd53 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:50.152797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3cd2576a-e101-4373-b6ff-8543eca5dd53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:50.183515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deb7438-e0a9-4087-9d43-0c40c74ef0cf None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:50.232978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd88fd-a624-4654-9cde-105428f15823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:50.233626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd88fd-a624-4654-9cde-105428f15823 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:50.233906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd88fd-a624-4654-9cde-105428f15823 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:50.234178 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdd88fd-a624-4654-9cde-105428f15823 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:50.235159 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd88fd-a624-4654-9cde-105428f15823 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:50.235449 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd88fd-a624-4654-9cde-105428f15823 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:50.237519 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdd88fd-a624-4654-9cde-105428f15823 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:50.251293 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 418/810] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:15:50.251293 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2576a-e101-4373-b6ff-8543eca5dd53 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:50.272567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 393/811] 10.222.0.39 () {56 vars in 1133 bytes} [Thu Jul 28 22:15:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:50.275823 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd88fd-a624-4654-9cde-105428f15823 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:50.316607 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:50.373567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:50.383320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd2576a-e101-4373-b6ff-8543eca5dd53 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:50.390340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdd88fd-a624-4654-9cde-105428f15823 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:50.423962 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c615d50-29c9-44b5-bbaf-ba97fd4d4337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:50.423962 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c615d50-29c9-44b5-bbaf-ba97fd4d4337 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:50.423962 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c615d50-29c9-44b5-bbaf-ba97fd4d4337 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:50.424447 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c615d50-29c9-44b5-bbaf-ba97fd4d4337 None admin] PATH_INFO: `/v3/services/network` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:50.425210 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c615d50-29c9-44b5-bbaf-ba97fd4d4337 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:50.426701 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c615d50-29c9-44b5-bbaf-ba97fd4d4337 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:50.430220 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ee278-7e34-4372-b63c-58c6d2dce1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:50.431018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ee278-7e34-4372-b63c-58c6d2dce1c5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:50.431456 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ee278-7e34-4372-b63c-58c6d2dce1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:50.431866 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ee278-7e34-4372-b63c-58c6d2dce1c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:50.436591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-2c615d50-29c9-44b5-bbaf-ba97fd4d4337 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 22:15:50.436591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:50.436591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:15:50.436591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:15:50.436591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:15:50.436591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:15:50.436591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:15:50.436591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:50.436591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 22:15:50.436591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:50.436591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:15:50.436591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:50.436591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:50.436591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:50.436591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:50.436591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:50.436591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:50.437743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:50.437743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:50.437743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:50.437743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:50.437743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:50.437743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:50.437743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:15:50.437743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:15:50.437743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:15:50.437743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:15:50.437743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:50.437743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:50.437743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:50.437743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:50.437743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:50.437743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:50.438787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:50.438787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:50.438787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:50.438787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:50.438787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:50.438787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:50.438787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:50.438787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:50.438787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:50.438787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:50.438787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:15:50.438787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:50.438787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 28 22:15:50.438787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:50.444704 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 419/812] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:50 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:50.445972 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c615d50-29c9-44b5-bbaf-ba97fd4d4337 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:50.446381 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ee278-7e34-4372-b63c-58c6d2dce1c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:50.451498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-906ee278-7e34-4372-b63c-58c6d2dce1c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:50.451498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:50.451498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:50.451498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:50.451498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:50.451498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:50.451498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:50.451498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:50.451498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:50.451498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:50.451498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:50.451498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:50.451498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:50.451498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:50.451498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:50.451498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:50.452393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:50.452393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:50.452393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:50.452393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:50.452393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:50.452393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:50.452393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:50.452393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:50.452393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:50.452393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:50.452393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:50.452393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:50.452393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:50.452393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:50.452393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:50.452393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:50.453230 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:50.453230 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:50.453230 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:50.453230 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:50.453230 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:50.453230 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:50.453230 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:50.453230 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:50.453230 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:50.453230 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:50.453230 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:50.453230 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:50.455823 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 394/813] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:50.456762 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ee278-7e34-4372-b63c-58c6d2dce1c5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:50.552071 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:50.558226 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c615d50-29c9-44b5-bbaf-ba97fd4d4337 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:50.561779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ee278-7e34-4372-b63c-58c6d2dce1c5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:50.594636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb7a28-6f79-4e19-90a5-3d379d7f932a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:50.595342 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb7a28-6f79-4e19-90a5-3d379d7f932a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:50.595725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb7a28-6f79-4e19-90a5-3d379d7f932a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:50.596090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb7a28-6f79-4e19-90a5-3d379d7f932a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:50.599409 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13924de4-5e3f-42ca-929e-e88de21f443a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:50.600217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13924de4-5e3f-42ca-929e-e88de21f443a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:50.600701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13924de4-5e3f-42ca-929e-e88de21f443a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:50.601169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13924de4-5e3f-42ca-929e-e88de21f443a None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:50.602671 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13924de4-5e3f-42ca-929e-e88de21f443a None admin] RBAC: Adding query filter params (name=network) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:50.603202 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13924de4-5e3f-42ca-929e-e88de21f443a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:50.604766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13924de4-5e3f-42ca-929e-e88de21f443a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:50.610589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 395/814] 10.222.0.39 () {56 vars in 1112 bytes} [Thu Jul 28 22:15:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:50.611638 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13924de4-5e3f-42ca-929e-e88de21f443a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:50.637749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:50.685135 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb7a28-6f79-4e19-90a5-3d379d7f932a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:50.686740 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb7a28-6f79-4e19-90a5-3d379d7f932a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:50.720771 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13924de4-5e3f-42ca-929e-e88de21f443a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:50.757112 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d2020-01b5-431e-8af7-30b671d3dded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:50.758312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d2020-01b5-431e-8af7-30b671d3dded None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:50.758816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d2020-01b5-431e-8af7-30b671d3dded None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:50.759304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87d2020-01b5-431e-8af7-30b671d3dded None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:50.760217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d2020-01b5-431e-8af7-30b671d3dded None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:50.760727 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d2020-01b5-431e-8af7-30b671d3dded None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:50.762313 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87d2020-01b5-431e-8af7-30b671d3dded None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:50.772208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 396/815] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:15:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:50.773378 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d2020-01b5-431e-8af7-30b671d3dded None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:50.795900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:50.805804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 420/816] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:50 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 22:15:50.806891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb7a28-6f79-4e19-90a5-3d379d7f932a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:50.883392 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:50.906799 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87d2020-01b5-431e-8af7-30b671d3dded None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:50.931524 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb7a28-6f79-4e19-90a5-3d379d7f932a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:50.944691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d6644-8fa4-4cbc-b3f9-8093c2c54ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:50.945721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d6644-8fa4-4cbc-b3f9-8093c2c54ebe None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:50.946074 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d6644-8fa4-4cbc-b3f9-8093c2c54ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:50.946421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13d6644-8fa4-4cbc-b3f9-8093c2c54ebe None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:50.947173 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d6644-8fa4-4cbc-b3f9-8093c2c54ebe None admin] RBAC: Adding query filter params (type=network) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:50.947559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d6644-8fa4-4cbc-b3f9-8093c2c54ebe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:50.949200 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13d6644-8fa4-4cbc-b3f9-8093c2c54ebe None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:50.954998 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 397/817] 10.222.0.39 () {56 vars in 1112 bytes} [Thu Jul 28 22:15:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:50.956092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d6644-8fa4-4cbc-b3f9-8093c2c54ebe None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:50.971099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6719c07-886d-4d84-9f6b-f1a201bd0a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:50.971744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6719c07-886d-4d84-9f6b-f1a201bd0a1e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:50.972025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6719c07-886d-4d84-9f6b-f1a201bd0a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:50.972294 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6719c07-886d-4d84-9f6b-f1a201bd0a1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:50.983148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6719c07-886d-4d84-9f6b-f1a201bd0a1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:50.984587 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6719c07-886d-4d84-9f6b-f1a201bd0a1e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:50.999735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-e6719c07-886d-4d84-9f6b-f1a201bd0a1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:50.999735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:50.999735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:50.999735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:50.999735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:50.999735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:50.999735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:50.999735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:50.999735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:50.999735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:50.999735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:50.999735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:50.999735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:50.999735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:50.999735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:51.001062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:51.001062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:51.001062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:51.001062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:51.001062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:51.001062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:51.001062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:51.001062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:51.001062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:51.001062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:51.001062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:51.001062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:51.001062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:51.001062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:51.001062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:51.001062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:51.001062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:51.001980 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:51.001980 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:51.001980 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:51.001980 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:51.005684 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 421/818] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:51.006541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6719c07-886d-4d84-9f6b-f1a201bd0a1e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:51.049607 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:51.072737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13d6644-8fa4-4cbc-b3f9-8093c2c54ebe None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:51.110868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6719c07-886d-4d84-9f6b-f1a201bd0a1e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:51.111250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc006c-6e95-4fde-bff1-80fa20763fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:51.111967 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc006c-6e95-4fde-bff1-80fa20763fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:51.112360 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc006c-6e95-4fde-bff1-80fa20763fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:51.112647 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccc006c-6e95-4fde-bff1-80fa20763fd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:51.122244 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:51.128633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc006c-6e95-4fde-bff1-80fa20763fd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:51.130136 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccc006c-6e95-4fde-bff1-80fa20763fd1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:51.148249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-dccc006c-6e95-4fde-bff1-80fa20763fd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:51.148249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:51.148249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:51.148249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:51.148249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:51.148249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:51.148249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:51.148249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:51.148249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:51.148249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:51.148249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:51.148249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:51.148249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:51.148249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:51.148249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:51.150721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:51.150721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:51.150721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:51.150721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:51.150721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:51.150721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:51.150721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:51.150721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:51.150721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:51.150721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:51.150721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:51.150721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:51.150721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:51.150721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:51.150721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:51.150721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:51.150721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:51.152792 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:51.152792 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:51.152792 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:51.152792 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:51.154764 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 398/819] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:51.155890 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc006c-6e95-4fde-bff1-80fa20763fd1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:51.160864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb30c7a-98e4-40f6-b260-5471c66bacbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:51.161822 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb30c7a-98e4-40f6-b260-5471c66bacbb None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:51.162299 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb30c7a-98e4-40f6-b260-5471c66bacbb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:51.162778 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb30c7a-98e4-40f6-b260-5471c66bacbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:51.163679 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb30c7a-98e4-40f6-b260-5471c66bacbb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:51.165351 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb30c7a-98e4-40f6-b260-5471c66bacbb None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:51.193802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 422/820] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:15:51 2022] POST /identity/v3/endpoints => generated 327 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:15:51.195119 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb30c7a-98e4-40f6-b260-5471c66bacbb None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:51.276654 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccc006c-6e95-4fde-bff1-80fa20763fd1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:51.315640 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:51.334614 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1728f-42c7-4d4b-b492-736521c38d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:51.335565 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1728f-42c7-4d4b-b492-736521c38d51 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:51.336084 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1728f-42c7-4d4b-b492-736521c38d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:51.336670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1728f-42c7-4d4b-b492-736521c38d51 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:51.337899 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1728f-42c7-4d4b-b492-736521c38d51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:51.338406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1728f-42c7-4d4b-b492-736521c38d51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:51.340075 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1728f-42c7-4d4b-b492-736521c38d51 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:51.366171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb30c7a-98e4-40f6-b260-5471c66bacbb None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:51.371106 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 399/821] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:51.378872 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1728f-42c7-4d4b-b492-736521c38d51 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:51.401590 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c95eb-a400-44c9-bbbf-36b9a7f5b8a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:51.402283 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c95eb-a400-44c9-bbbf-36b9a7f5b8a9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:51.402565 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c95eb-a400-44c9-bbbf-36b9a7f5b8a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:51.402846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66c95eb-a400-44c9-bbbf-36b9a7f5b8a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:51.403552 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c95eb-a400-44c9-bbbf-36b9a7f5b8a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:51.403904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c95eb-a400-44c9-bbbf-36b9a7f5b8a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:51.409739 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66c95eb-a400-44c9-bbbf-36b9a7f5b8a9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:51.420076 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 423/822] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:51.426515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c95eb-a400-44c9-bbbf-36b9a7f5b8a9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:51.471252 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:51.561360 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66c95eb-a400-44c9-bbbf-36b9a7f5b8a9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:51.577154 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1728f-42c7-4d4b-b492-736521c38d51 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:51.604741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31aad67-8182-485e-88a8-c8e32da79ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:51.605808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31aad67-8182-485e-88a8-c8e32da79ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:51.606260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31aad67-8182-485e-88a8-c8e32da79ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:51.606695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31aad67-8182-485e-88a8-c8e32da79ab1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:51.614563 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31aad67-8182-485e-88a8-c8e32da79ab1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:51.616192 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31aad67-8182-485e-88a8-c8e32da79ab1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:51.622852 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f39d1-ea18-4d55-affc-01fdf4e02773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:51.623860 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f39d1-ea18-4d55-affc-01fdf4e02773 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:51.624349 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f39d1-ea18-4d55-affc-01fdf4e02773 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:51.624841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f39d1-ea18-4d55-affc-01fdf4e02773 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:51.630328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-f31aad67-8182-485e-88a8-c8e32da79ab1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:51.630328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:51.630328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:51.630328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:51.630328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:51.630328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:51.630328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:51.630328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:51.630328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:51.630328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:51.630328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:51.630328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:51.630328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:51.630328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:51.630328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:51.631236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:51.631236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:51.631236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:51.631236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:51.631236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:51.631236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:51.631236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:51.631236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:51.631236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:51.631236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:51.631236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:51.631236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:51.631236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:51.631236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:51.631236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:51.631236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:51.631236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:51.637869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:51.637869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:51.637869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:51.637869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:51.637869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:51.637869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:51.637869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:51.637869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:51.637869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 424/823] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:15:51.637869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d4375-124a-4862-9a9e-d4370072f242 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:51.637869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d4375-124a-4862-9a9e-d4370072f242 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:51.638482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d4375-124a-4862-9a9e-d4370072f242 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:51.638555 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f39d1-ea18-4d55-affc-01fdf4e02773 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:51.640068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f39d1-ea18-4d55-affc-01fdf4e02773 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:51.640174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 425/824] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:51.640912 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d4375-124a-4862-9a9e-d4370072f242 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:51.654352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-2b2f39d1-ea18-4d55-affc-01fdf4e02773 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 22:15:51.654352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:51.654352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:51.654352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:51.654352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:51.654352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:51.654352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:51.654352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:51.654352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:51.654352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:51.654352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:51.654352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:51.654352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:51.654352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:51.654352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:51.654352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:51.655361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:51.655361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:51.655361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:51.655361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:51.655361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:51.655361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:51.655361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:51.655361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:51.655361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:51.655361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:51.655361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:51.655361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:51.655361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:51.655361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:51.655361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:51.655361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:51.655361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:51.655361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:51.656754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:51.656754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:51.656754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:51.656754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:51.656754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:51.656754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:51.656754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:51.656754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:51.656754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 22:15:51.656754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:51.656754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 400/825] 10.222.0.39 () {56 vars in 1117 bytes} [Thu Jul 28 22:15:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:15:51.657379 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0e6d9-9868-4433-8938-2296bd91738e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:51.657379 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0e6d9-9868-4433-8938-2296bd91738e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:51.661582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0e6d9-9868-4433-8938-2296bd91738e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:51.682458 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a5b0e6d9-9868-4433-8938-2296bd91738e None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:51.684025 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a5b0e6d9-9868-4433-8938-2296bd91738e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:51.733083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:51.737192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:51.751370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329d4375-124a-4862-9a9e-d4370072f242 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:51.766640 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 401/826] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:15:51.768587 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0e6d9-9868-4433-8938-2296bd91738e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:51.788501 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440440fe-c6f6-42db-a2e1-563171838f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:51.789147 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440440fe-c6f6-42db-a2e1-563171838f2e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:51.789992 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440440fe-c6f6-42db-a2e1-563171838f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:51.790267 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440440fe-c6f6-42db-a2e1-563171838f2e None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:51.791024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440440fe-c6f6-42db-a2e1-563171838f2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:51.791335 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440440fe-c6f6-42db-a2e1-563171838f2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:51.792774 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440440fe-c6f6-42db-a2e1-563171838f2e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:51.804309 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 426/827] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:51.806650 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7dd43c-eec4-4fe4-9a21-91898d4f3d48 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:51.806934 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7dd43c-eec4-4fe4-9a21-91898d4f3d48 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:51.807211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7dd43c-eec4-4fe4-9a21-91898d4f3d48 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:51.808717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 427/828] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:51.809733 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7dd43c-eec4-4fe4-9a21-91898d4f3d48 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:51.874664 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0e6d9-9868-4433-8938-2296bd91738e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:51.909121 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59394dd5-e4e6-4a2e-987a-75611d4f0e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:51.909760 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59394dd5-e4e6-4a2e-987a-75611d4f0e27 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:51.910039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59394dd5-e4e6-4a2e-987a-75611d4f0e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:51.910314 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59394dd5-e4e6-4a2e-987a-75611d4f0e27 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:51.911103 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59394dd5-e4e6-4a2e-987a-75611d4f0e27 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:51.911408 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59394dd5-e4e6-4a2e-987a-75611d4f0e27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:51.912834 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59394dd5-e4e6-4a2e-987a-75611d4f0e27 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:51.914009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7dd43c-eec4-4fe4-9a21-91898d4f3d48 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:51.930860 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 402/829] 10.222.0.39 () {56 vars in 1149 bytes} [Thu Jul 28 22:15:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:51.931631 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59394dd5-e4e6-4a2e-987a-75611d4f0e27 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:51.956464 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b27f9-94d2-4df8-a879-a0b05b6c36b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:51.957102 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b27f9-94d2-4df8-a879-a0b05b6c36b1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:51.957382 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b27f9-94d2-4df8-a879-a0b05b6c36b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:51.957695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b27f9-94d2-4df8-a879-a0b05b6c36b1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:51.958321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b27f9-94d2-4df8-a879-a0b05b6c36b1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:51.959696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b27f9-94d2-4df8-a879-a0b05b6c36b1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:51.969064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-8b2b27f9-94d2-4df8-a879-a0b05b6c36b1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 22:15:51.969064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:51.969064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:15:51.969064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:15:51.969064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:15:51.969064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:15:51.969064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:15:51.969064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:51.969064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 22:15:51.969064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:51.969064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:15:51.969064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:51.969064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:51.969064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:51.969064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:51.969064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:51.969064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:51.970260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:51.970260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:51.970260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:51.970260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:51.970260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:51.970260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:51.970260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:15:51.970260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:15:51.970260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:15:51.970260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:15:51.970260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:51.970260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:51.970260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:51.970260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:51.970260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:51.970260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:51.971115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:51.971115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:51.971115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:51.971115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:51.971115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:51.971115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:51.971115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:51.971115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:51.971115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:51.971115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:51.971115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:15:51.971115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:51.971115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 22:15:51.971115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:51.971115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 428/830] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:15:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:51.971928 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcf06e-bd9f-4566-801c-230076595265 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:51.971928 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcf06e-bd9f-4566-801c-230076595265 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:51.972061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcf06e-bd9f-4566-801c-230076595265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:51.980812 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:51.991934 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-61fcf06e-bd9f-4566-801c-230076595265 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:51.993205 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-61fcf06e-bd9f-4566-801c-230076595265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:51.995496 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:52.043375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59394dd5-e4e6-4a2e-987a-75611d4f0e27 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:52.061476 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 429/831] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:15:52.062328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcf06e-bd9f-4566-801c-230076595265 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:52.079762 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c0a7f-dadf-4096-9cd4-e3c4ae6aaa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:52.080275 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c0a7f-dadf-4096-9cd4-e3c4ae6aaa81 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:52.080553 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c0a7f-dadf-4096-9cd4-e3c4ae6aaa81 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:52.080821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8c0a7f-dadf-4096-9cd4-e3c4ae6aaa81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:52.081637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c0a7f-dadf-4096-9cd4-e3c4ae6aaa81 None admin] RBAC: Adding query filter params (role.id=aded7bd43e974686b3c23bd9d4255b24, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=77efe8046ea0468cb0c7c58c24fc4e75) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:52.081944 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c0a7f-dadf-4096-9cd4-e3c4ae6aaa81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:52.083359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8c0a7f-dadf-4096-9cd4-e3c4ae6aaa81 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:52.090322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 403/832] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:15:51 2022] GET /identity/v3/role_assignments?user.id=77efe8046ea0468cb0c7c58c24fc4e75&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=aded7bd43e974686b3c23bd9d4255b24 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:52.101554 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c0a7f-dadf-4096-9cd4-e3c4ae6aaa81 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:52.182259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcf06e-bd9f-4566-801c-230076595265 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:52.214866 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8c0a7f-dadf-4096-9cd4-e3c4ae6aaa81 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:52.216863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:52.224630 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a3045-742c-4fe1-8c09-effd7702e4af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:52.225289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a3045-742c-4fe1-8c09-effd7702e4af None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:52.225594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a3045-742c-4fe1-8c09-effd7702e4af None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:52.226200 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a3045-742c-4fe1-8c09-effd7702e4af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:52.234274 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:52.238604 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a3045-742c-4fe1-8c09-effd7702e4af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:52.240730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822a3045-742c-4fe1-8c09-effd7702e4af None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:52.259894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-822a3045-742c-4fe1-8c09-effd7702e4af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:52.259894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:52.259894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:52.259894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:52.259894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:52.259894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:52.259894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:52.259894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:52.259894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:52.259894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:52.259894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:52.259894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:52.259894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:52.259894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:52.259894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:52.260828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:52.260828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:52.260828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:52.260828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:52.260828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:52.260828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:52.260828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:52.260828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:52.260828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:52.260828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:52.260828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:52.260828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:52.260828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:52.260828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:52.260828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:52.260828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:52.260828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:52.261761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:52.261761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:52.261761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:52.261761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:52.261761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 430/833] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:52.262477 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a3045-742c-4fe1-8c09-effd7702e4af None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:52.264050 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38908e-cef2-4b05-9e89-6bf41524c1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:52.264691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38908e-cef2-4b05-9e89-6bf41524c1de None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:52.265146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38908e-cef2-4b05-9e89-6bf41524c1de None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:52.265440 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38908e-cef2-4b05-9e89-6bf41524c1de None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:52.267144 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38908e-cef2-4b05-9e89-6bf41524c1de None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:52.267357 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38908e-cef2-4b05-9e89-6bf41524c1de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:52.268749 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38908e-cef2-4b05-9e89-6bf41524c1de None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:52.274382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 404/834] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:15:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:52.275586 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38908e-cef2-4b05-9e89-6bf41524c1de None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:52.374753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a3045-742c-4fe1-8c09-effd7702e4af None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:52.395278 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38908e-cef2-4b05-9e89-6bf41524c1de None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:52.409461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40edba-d63f-4303-a9c4-d37a7a78b8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:52.410055 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40edba-d63f-4303-a9c4-d37a7a78b8ce None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:52.410342 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40edba-d63f-4303-a9c4-d37a7a78b8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:52.410619 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40edba-d63f-4303-a9c4-d37a7a78b8ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:52.421329 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c40edba-d63f-4303-a9c4-d37a7a78b8ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:52.426922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-1c40edba-d63f-4303-a9c4-d37a7a78b8ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:15:52.426922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:52.426922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:52.426922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:52.426922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:52.426922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:52.426922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:52.426922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:52.426922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:52.426922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:52.426922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:52.426922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:52.426922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:52.426922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:52.426922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:52.426922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:52.428090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:52.428090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:52.428090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:52.428090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:52.428090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:52.428090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:52.428090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:52.428090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:52.428090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:52.428090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:52.428090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:52.428090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:52.428090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:52.428090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:52.428090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:52.428090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:52.428943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:52.428943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:52.428943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:52.428943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:52.428943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:52.428943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:52.428943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:52.428943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:52.428943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:52.428943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:52.428943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:15:52.428943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:52.428943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 431/835] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:15:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:52.429740 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40edba-d63f-4303-a9c4-d37a7a78b8ce None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:52.441265 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0307a8-e1a8-4703-9c8c-c902070dd14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:52.441928 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0307a8-e1a8-4703-9c8c-c902070dd14a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:52.442213 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0307a8-e1a8-4703-9c8c-c902070dd14a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:52.442491 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0307a8-e1a8-4703-9c8c-c902070dd14a None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:52.443200 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0307a8-e1a8-4703-9c8c-c902070dd14a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:52.443631 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0307a8-e1a8-4703-9c8c-c902070dd14a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:52.445134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0307a8-e1a8-4703-9c8c-c902070dd14a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:52.458278 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 405/836] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:52.459226 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0307a8-e1a8-4703-9c8c-c902070dd14a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:52.488853 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:52.491481 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:52.546337 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40edba-d63f-4303-a9c4-d37a7a78b8ce None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:52.569039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0307a8-e1a8-4703-9c8c-c902070dd14a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:52.582531 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c952d3-6bc8-4c2b-b387-ad4a2074023a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:52.583159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c952d3-6bc8-4c2b-b387-ad4a2074023a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:52.583445 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c952d3-6bc8-4c2b-b387-ad4a2074023a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:52.583727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c952d3-6bc8-4c2b-b387-ad4a2074023a None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:52.584419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c952d3-6bc8-4c2b-b387-ad4a2074023a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:52.584725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c952d3-6bc8-4c2b-b387-ad4a2074023a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:52.586652 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c952d3-6bc8-4c2b-b387-ad4a2074023a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:52.592471 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 432/837] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:15:52 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:52.593279 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c952d3-6bc8-4c2b-b387-ad4a2074023a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:52.619706 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0006fbc-6fb7-4208-9788-5f14583f1651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:52.620998 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0006fbc-6fb7-4208-9788-5f14583f1651 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:52.622091 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0006fbc-6fb7-4208-9788-5f14583f1651 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:52.622567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0006fbc-6fb7-4208-9788-5f14583f1651 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:52.623800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0006fbc-6fb7-4208-9788-5f14583f1651 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:52.624589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0006fbc-6fb7-4208-9788-5f14583f1651 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:52.626744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0006fbc-6fb7-4208-9788-5f14583f1651 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:52.640639 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 406/838] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:15:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:52.646278 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0006fbc-6fb7-4208-9788-5f14583f1651 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:52.727784 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c952d3-6bc8-4c2b-b387-ad4a2074023a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:52.745472 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:52.775649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a919b-63ba-4749-b60a-1e35b366d92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:52.776300 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a919b-63ba-4749-b60a-1e35b366d92f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:52.776580 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a919b-63ba-4749-b60a-1e35b366d92f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:52.776898 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a919b-63ba-4749-b60a-1e35b366d92f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:52.781176 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:52.785158 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a919b-63ba-4749-b60a-1e35b366d92f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:52.786646 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a919b-63ba-4749-b60a-1e35b366d92f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:52.797699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0006fbc-6fb7-4208-9788-5f14583f1651 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:52.799419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-b36a919b-63ba-4749-b60a-1e35b366d92f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:52.799419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:52.799419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:52.799419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:52.799419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:52.799419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:52.799419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:52.799419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:52.799419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:52.799419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:52.799419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:52.799419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:52.799419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:52.799419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:52.799419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:52.800280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:52.800280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:52.800280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:52.800280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:52.800280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:52.800280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:52.800280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:52.800280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:52.800280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:52.800280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:52.800280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:52.800280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:52.800280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:52.800280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:52.800280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:52.800280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:52.800280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:52.801190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:52.801190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:52.801190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:52.801190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:52.801190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:52.801190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:52.801190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:52.801190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:52.801190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 433/839] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:52.806262 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a919b-63ba-4749-b60a-1e35b366d92f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:52.843695 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855eb77c-e53c-4bb7-8753-edeb35e0ea67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:52.844474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855eb77c-e53c-4bb7-8753-edeb35e0ea67 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:52.844757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855eb77c-e53c-4bb7-8753-edeb35e0ea67 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:52.845037 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855eb77c-e53c-4bb7-8753-edeb35e0ea67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:52.859741 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855eb77c-e53c-4bb7-8753-edeb35e0ea67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:52.861493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855eb77c-e53c-4bb7-8753-edeb35e0ea67 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:52.882685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-855eb77c-e53c-4bb7-8753-edeb35e0ea67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:52.882685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:52.882685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:52.882685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:52.882685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:52.882685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:52.882685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:52.882685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:52.882685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:52.882685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:52.882685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:52.882685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:52.882685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:52.882685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:52.882685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:52.884248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:52.884248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:52.884248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:52.884248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:52.884248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:52.884248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:52.884248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:52.884248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:52.884248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:52.884248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:52.884248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:52.884248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:52.884248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:52.884248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:52.884248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:52.884248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:52.884248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:52.885155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:52.885155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:52.885155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:52.885155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:52.889855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 407/840] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:15:52.891430 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855eb77c-e53c-4bb7-8753-edeb35e0ea67 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:52.927597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a919b-63ba-4749-b60a-1e35b366d92f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:52.964744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ced07-b0f4-4deb-bb51-8a8daaed6383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:52.965673 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ced07-b0f4-4deb-bb51-8a8daaed6383 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:52.965963 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ced07-b0f4-4deb-bb51-8a8daaed6383 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:52.966248 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ced07-b0f4-4deb-bb51-8a8daaed6383 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:52.967007 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ced07-b0f4-4deb-bb51-8a8daaed6383 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:52.967323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ced07-b0f4-4deb-bb51-8a8daaed6383 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:52.969573 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ced07-b0f4-4deb-bb51-8a8daaed6383 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:52.985603 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 434/841] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:52.990856 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ced07-b0f4-4deb-bb51-8a8daaed6383 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:53.012754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855eb77c-e53c-4bb7-8753-edeb35e0ea67 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:53.038052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:53.057701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b00d0-6df5-46b4-bf04-b6b945325aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:53.058335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b00d0-6df5-46b4-bf04-b6b945325aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:53.058619 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b00d0-6df5-46b4-bf04-b6b945325aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:53.059291 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b00d0-6df5-46b4-bf04-b6b945325aa9 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:53.060075 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b00d0-6df5-46b4-bf04-b6b945325aa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:53.060557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b00d0-6df5-46b4-bf04-b6b945325aa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:53.062258 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04b00d0-6df5-46b4-bf04-b6b945325aa9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:53.074660 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:53.079309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 408/842] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:53.087489 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b00d0-6df5-46b4-bf04-b6b945325aa9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:53.124779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ced07-b0f4-4deb-bb51-8a8daaed6383 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:53.168180 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15949e-4ecd-40e8-b31a-3d4084d6a88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:53.168867 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15949e-4ecd-40e8-b31a-3d4084d6a88b None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:53.169150 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15949e-4ecd-40e8-b31a-3d4084d6a88b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:53.169429 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15949e-4ecd-40e8-b31a-3d4084d6a88b None admin] PATH_INFO: `/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/85626350279c40ea99da93b9ed6994b2/roles/a66ab77a63174592a1e468d2464e8bc4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:53.194862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15949e-4ecd-40e8-b31a-3d4084d6a88b None admin] RBAC: Authorizing `identity:create_grant(project_id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user_id=85626350279c40ea99da93b9ed6994b2, role_id=a66ab77a63174592a1e468d2464e8bc4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:53.197105 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d15949e-4ecd-40e8-b31a-3d4084d6a88b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:53.225554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 435/843] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:15:52 2022] PUT /identity/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/85626350279c40ea99da93b9ed6994b2/roles/a66ab77a63174592a1e468d2464e8bc4 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:15:53.233191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b00d0-6df5-46b4-bf04-b6b945325aa9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:53.276346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2924935-a120-4c7f-bc7c-b890e704e6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:53.276937 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2924935-a120-4c7f-bc7c-b890e704e6ad None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:53.277265 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2924935-a120-4c7f-bc7c-b890e704e6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:53.277567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2924935-a120-4c7f-bc7c-b890e704e6ad None admin] PATH_INFO: `/v3/users/swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:53.286435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2924935-a120-4c7f-bc7c-b890e704e6ad None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:53.287917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2924935-a120-4c7f-bc7c-b890e704e6ad None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:53.307667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-f2924935-a120-4c7f-bc7c-b890e704e6ad None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 28 22:15:53.307667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:53.307667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:53.307667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:53.307667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:53.307667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:53.307667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:53.307667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:53.307667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:53.307667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:53.307667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:53.307667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:53.307667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:53.307667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:53.307667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:53.307667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:53.308713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:53.308713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:53.308713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:53.308713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:53.308713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:53.308713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:53.308713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:53.308713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:53.308713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:53.308713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:53.308713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:53.308713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:53.308713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:53.308713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:53.308713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:53.308713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:53.308713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:53.308713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:53.309697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:53.309697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:53.309697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:53.309697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:53.309697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:53.309697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:53.309697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:53.309697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:53.309697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 28 22:15:53.309697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:53.309697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 409/844] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:15:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:53.315158 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15949e-4ecd-40e8-b31a-3d4084d6a88b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:53.413649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:53.433222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15949e-4ecd-40e8-b31a-3d4084d6a88b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:53.470486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e694228-0483-4e55-9bed-9cd4f11a7441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:53.471174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e694228-0483-4e55-9bed-9cd4f11a7441 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:53.471458 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e694228-0483-4e55-9bed-9cd4f11a7441 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:53.471735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e694228-0483-4e55-9bed-9cd4f11a7441 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:53.472584 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e694228-0483-4e55-9bed-9cd4f11a7441 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:53.472899 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e694228-0483-4e55-9bed-9cd4f11a7441 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:53.474375 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e694228-0483-4e55-9bed-9cd4f11a7441 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:53.493275 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 436/845] 10.222.0.39 () {56 vars in 1135 bytes} [Thu Jul 28 22:15:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:53.498878 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2924935-a120-4c7f-bc7c-b890e704e6ad None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:53.562358 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:53.607498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2924935-a120-4c7f-bc7c-b890e704e6ad None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:53.643479 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1fc8a-702a-4ecc-a940-ba42d81ca66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:53.644101 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1fc8a-702a-4ecc-a940-ba42d81ca66e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:53.644384 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1fc8a-702a-4ecc-a940-ba42d81ca66e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:53.644658 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab1fc8a-702a-4ecc-a940-ba42d81ca66e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:53.715841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1fc8a-702a-4ecc-a940-ba42d81ca66e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:53.717305 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab1fc8a-702a-4ecc-a940-ba42d81ca66e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:53.782176 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:53.827641 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 410/846] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:53 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 22:15:53.833257 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e694228-0483-4e55-9bed-9cd4f11a7441 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:53.939902 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e694228-0483-4e55-9bed-9cd4f11a7441 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:53.981343 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1bfae-4356-44d9-add6-2be0946fc7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:53.982419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1bfae-4356-44d9-add6-2be0946fc7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:53.982902 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1bfae-4356-44d9-add6-2be0946fc7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:53.983400 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1bfae-4356-44d9-add6-2be0946fc7e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:53.994174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f1bfae-4356-44d9-add6-2be0946fc7e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:53.995856 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f1bfae-4356-44d9-add6-2be0946fc7e2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:53.997764 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:54.012116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-32f1bfae-4356-44d9-add6-2be0946fc7e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:54.012116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:54.012116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:54.012116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:54.012116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:54.012116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:54.012116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:54.012116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:54.012116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:54.012116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:54.012116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:54.012116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:54.012116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:54.012116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:54.012116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:54.013066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:54.013066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:54.013066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:54.013066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:54.013066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:54.013066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:54.013066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:54.013066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:54.013066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:54.013066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:54.013066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:54.013066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:54.013066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:54.013066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:54.013066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:54.013066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:54.013066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:54.014268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:54.014268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:54.014268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:54.014268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:54.016409 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 437/847] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:54.022091 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1fc8a-702a-4ecc-a940-ba42d81ca66e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:54.126851 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab1fc8a-702a-4ecc-a940-ba42d81ca66e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:54.163445 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e1d9b-1c97-4050-b734-36fb36215b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:54.164376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e1d9b-1c97-4050-b734-36fb36215b94 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:54.164666 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e1d9b-1c97-4050-b734-36fb36215b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:54.164942 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e1d9b-1c97-4050-b734-36fb36215b94 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:54.165677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e1d9b-1c97-4050-b734-36fb36215b94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:54.165986 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e1d9b-1c97-4050-b734-36fb36215b94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:54.167370 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e1d9b-1c97-4050-b734-36fb36215b94 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:54.179333 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 411/848] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:54.184501 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00238d8e-bcb0-406b-8bea-47da812579ff None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:54.184958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00238d8e-bcb0-406b-8bea-47da812579ff None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:54.185061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00238d8e-bcb0-406b-8bea-47da812579ff None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:54.186649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 438/849] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:54.189311 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e1d9b-1c97-4050-b734-36fb36215b94 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:54.193677 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecc5b5-95f6-4947-9b62-59b835eb9f3f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:54.194179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecc5b5-95f6-4947-9b62-59b835eb9f3f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:54.194877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecc5b5-95f6-4947-9b62-59b835eb9f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:54.217705 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c9ecc5b5-95f6-4947-9b62-59b835eb9f3f None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:54.219682 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c9ecc5b5-95f6-4947-9b62-59b835eb9f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:54.229944 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:54.305595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 439/850] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:15:54.321644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e1d9b-1c97-4050-b734-36fb36215b94 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:54.329077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecc5b5-95f6-4947-9b62-59b835eb9f3f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:54.366794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717923f-77d3-4250-8070-5b4f1c7e5fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:54.369110 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717923f-77d3-4250-8070-5b4f1c7e5fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:54.369631 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717923f-77d3-4250-8070-5b4f1c7e5fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:54.370133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0717923f-77d3-4250-8070-5b4f1c7e5fc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:54.379059 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0717923f-77d3-4250-8070-5b4f1c7e5fc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:54.381519 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0717923f-77d3-4250-8070-5b4f1c7e5fc2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:54.397374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-0717923f-77d3-4250-8070-5b4f1c7e5fc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:54.397374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:54.397374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:54.397374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:54.397374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:54.397374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:54.397374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:54.397374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:54.397374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:54.397374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:54.397374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:54.397374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:54.397374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:54.397374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:54.397374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:54.398808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:54.398808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:54.398808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:54.398808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:54.398808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:54.398808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:54.398808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:54.398808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:54.398808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:54.398808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:54.398808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:54.398808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:54.398808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:54.398808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:54.398808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:54.398808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:54.398808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:54.399927 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:54.399927 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:54.399927 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:54.399927 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:54.399927 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:54.399927 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:54.399927 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:54.399927 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:54.402176 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 412/851] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:54.410113 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717923f-77d3-4250-8070-5b4f1c7e5fc2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:54.443728 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ecc5b5-95f6-4947-9b62-59b835eb9f3f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:54.470714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:54.485853 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedab5f-1178-4e2f-8a24-4656dc85e5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:54.486345 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedab5f-1178-4e2f-8a24-4656dc85e5fa None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:54.486718 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedab5f-1178-4e2f-8a24-4656dc85e5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:54.487086 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bedab5f-1178-4e2f-8a24-4656dc85e5fa None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:54.487809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedab5f-1178-4e2f-8a24-4656dc85e5fa None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:54.489276 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bedab5f-1178-4e2f-8a24-4656dc85e5fa None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:54.499596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-6bedab5f-1178-4e2f-8a24-4656dc85e5fa None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 22:15:54.499596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:54.499596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:15:54.499596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:15:54.499596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:15:54.499596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:15:54.499596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:15:54.499596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:54.499596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 22:15:54.499596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:54.499596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:15:54.499596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:54.499596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:54.499596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:54.499596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:54.499596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:54.500935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:54.500935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:54.500935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:54.500935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:54.500935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:54.500935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:54.500935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:54.500935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:15:54.500935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:15:54.500935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:15:54.500935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:15:54.500935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:54.500935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:54.500935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:54.500935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:54.500935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:54.502027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:54.502027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:54.502027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:54.502027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:54.502027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:54.502027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:54.502027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:54.502027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:54.502027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:54.502027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:54.502027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:54.502027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:15:54.502027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:54.502027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 22:15:54.502027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:54.508935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 440/852] 10.222.0.39 () {56 vars in 1132 bytes} [Thu Jul 28 22:15:54 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:15:54.512798 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedab5f-1178-4e2f-8a24-4656dc85e5fa None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:54.517983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:54.544522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0717923f-77d3-4250-8070-5b4f1c7e5fc2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:54.589591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf073c4-4d18-41fd-a4a5-728da61d5649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:54.590430 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf073c4-4d18-41fd-a4a5-728da61d5649 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:54.590735 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf073c4-4d18-41fd-a4a5-728da61d5649 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:54.591007 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf073c4-4d18-41fd-a4a5-728da61d5649 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:54.592176 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf073c4-4d18-41fd-a4a5-728da61d5649 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:54.592497 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf073c4-4d18-41fd-a4a5-728da61d5649 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:54.594218 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf073c4-4d18-41fd-a4a5-728da61d5649 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:54.617014 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 413/853] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:54.618557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf073c4-4d18-41fd-a4a5-728da61d5649 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:54.649084 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bedab5f-1178-4e2f-8a24-4656dc85e5fa None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:54.683974 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74296559-0dce-42ce-8b95-84a9f8d9a234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:54.684597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74296559-0dce-42ce-8b95-84a9f8d9a234 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:54.684921 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74296559-0dce-42ce-8b95-84a9f8d9a234 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:54.685199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74296559-0dce-42ce-8b95-84a9f8d9a234 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:54.685918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74296559-0dce-42ce-8b95-84a9f8d9a234 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:54.686222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74296559-0dce-42ce-8b95-84a9f8d9a234 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:54.687579 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74296559-0dce-42ce-8b95-84a9f8d9a234 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:54.697415 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 441/854] 10.222.0.39 () {56 vars in 1126 bytes} [Thu Jul 28 22:15:54 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:54.698641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74296559-0dce-42ce-8b95-84a9f8d9a234 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:54.739281 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf073c4-4d18-41fd-a4a5-728da61d5649 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:54.761327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:54.779422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b26d7-2dbf-4759-b0f9-41f1056a4212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:54.780043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b26d7-2dbf-4759-b0f9-41f1056a4212 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:54.780327 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b26d7-2dbf-4759-b0f9-41f1056a4212 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:54.780603 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b26d7-2dbf-4759-b0f9-41f1056a4212 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:54.781395 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b26d7-2dbf-4759-b0f9-41f1056a4212 None admin] RBAC: Adding query filter params (role.id=aded7bd43e974686b3c23bd9d4255b24, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=e3c9107872f147f6ab8698b67bb545af) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:54.781746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b26d7-2dbf-4759-b0f9-41f1056a4212 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:54.783179 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b26d7-2dbf-4759-b0f9-41f1056a4212 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:54.789223 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 414/855] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:15:54 2022] GET /identity/v3/role_assignments?user.id=e3c9107872f147f6ab8698b67bb545af&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=aded7bd43e974686b3c23bd9d4255b24 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:54.790488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1969a2-bb07-41ca-9636-52ee6cc0982a None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:54.790780 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1969a2-bb07-41ca-9636-52ee6cc0982a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:54.791060 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1969a2-bb07-41ca-9636-52ee6cc0982a None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:54.792428 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 415/856] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:54.797858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91177a-04bc-4164-a48a-2a877cbdd587 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:54.798148 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91177a-04bc-4164-a48a-2a877cbdd587 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:54.798432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91177a-04bc-4164-a48a-2a877cbdd587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:54.806694 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:54.820725 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2e91177a-04bc-4164-a48a-2a877cbdd587 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:54.822535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2e91177a-04bc-4164-a48a-2a877cbdd587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:54.825869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74296559-0dce-42ce-8b95-84a9f8d9a234 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:54.870537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9cc83-2b83-4b9c-b8ca-a21ec0692fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:54.871548 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9cc83-2b83-4b9c-b8ca-a21ec0692fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:54.872026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9cc83-2b83-4b9c-b8ca-a21ec0692fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:54.872448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9cc83-2b83-4b9c-b8ca-a21ec0692fa1 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:54.873300 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de9cc83-2b83-4b9c-b8ca-a21ec0692fa1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:54.873898 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de9cc83-2b83-4b9c-b8ca-a21ec0692fa1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:54.875697 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de9cc83-2b83-4b9c-b8ca-a21ec0692fa1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:54.890053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 442/857] 10.222.0.39 () {56 vars in 1126 bytes} [Thu Jul 28 22:15:54 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:54.905235 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 416/858] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:15:54.929509 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9cc83-2b83-4b9c-b8ca-a21ec0692fa1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:55.051403 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9cc83-2b83-4b9c-b8ca-a21ec0692fa1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:55.069985 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:55.090906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c36460-d6c5-4dbe-9d96-3466e5aff169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:55.091551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c36460-d6c5-4dbe-9d96-3466e5aff169 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:55.091830 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c36460-d6c5-4dbe-9d96-3466e5aff169 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:55.092101 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c36460-d6c5-4dbe-9d96-3466e5aff169 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:55.092790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c36460-d6c5-4dbe-9d96-3466e5aff169 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:55.094184 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c36460-d6c5-4dbe-9d96-3466e5aff169 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:55.111829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 443/859] 10.222.0.39 () {60 vars in 1141 bytes} [Thu Jul 28 22:15:54 2022] POST /identity/v3/services => generated 251 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:15:55.438046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ee47c-b152-4087-be20-a16bd81561b2 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:55.438597 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ee47c-b152-4087-be20-a16bd81561b2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:55.438597 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8ee47c-b152-4087-be20-a16bd81561b2 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:55.440215 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 417/860] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:55.445506 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167d6e6-f8f0-4d0f-9d1a-9333737d7a8a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:55.446025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167d6e6-f8f0-4d0f-9d1a-9333737d7a8a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:55.446103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4167d6e6-f8f0-4d0f-9d1a-9333737d7a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:55.465504 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-4167d6e6-f8f0-4d0f-9d1a-9333737d7a8a None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:55.466746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4167d6e6-f8f0-4d0f-9d1a-9333737d7a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:55.541309 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 444/861] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:55 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:15:55.565230 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ee47c-b152-4087-be20-a16bd81561b2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:55.672194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8ee47c-b152-4087-be20-a16bd81561b2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:55.707719 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe60a80-f522-4352-bb52-a8cb38304690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:55.708276 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe60a80-f522-4352-bb52-a8cb38304690 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:55.708558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe60a80-f522-4352-bb52-a8cb38304690 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:55.708827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe60a80-f522-4352-bb52-a8cb38304690 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:55.711272 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:55.721154 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe60a80-f522-4352-bb52-a8cb38304690 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:15:55.726248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-cbe60a80-f522-4352-bb52-a8cb38304690 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:55.726248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:55.726248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:55.726248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:55.726248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:55.726248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:55.726248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:55.726248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:55.726248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:55.726248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:55.726248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:55.726248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:55.726248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:15:55.726248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:15:55.726248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:15:55.726248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:15:55.728874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:15:55.728874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:15:55.728874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:15:55.728874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:15:55.728874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:55.728874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:55.728874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:55.728874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:55.728874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:55.728874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:55.728874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:55.728874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:55.728874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:55.728874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:55.728874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:55.728874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:55.730766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:55.730766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:55.730766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:55.730766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:55.730766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:15:55.730766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:15:55.730766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:15:55.730766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:15:55.730766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:15:55.730766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:15:55.730766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:15:55.730766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:55.733273 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 418/862] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:15:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:55.742808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167d6e6-f8f0-4d0f-9d1a-9333737d7a8a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:55.850583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4167d6e6-f8f0-4d0f-9d1a-9333737d7a8a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:55.852407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:55.886624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cee394-70f8-42b1-a940-775d61390abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:55.887438 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cee394-70f8-42b1-a940-775d61390abe None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:55.887778 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cee394-70f8-42b1-a940-775d61390abe None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:55.888346 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cee394-70f8-42b1-a940-775d61390abe None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:55.889326 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cee394-70f8-42b1-a940-775d61390abe None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:55.889656 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cee394-70f8-42b1-a940-775d61390abe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:55.891028 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cee394-70f8-42b1-a940-775d61390abe None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:55.900398 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 445/863] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:15:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:55.905553 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe60a80-f522-4352-bb52-a8cb38304690 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:56.011883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe60a80-f522-4352-bb52-a8cb38304690 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:56.047698 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0618d54-c1c4-4dca-b57b-5c8ae716d35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:56.048347 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0618d54-c1c4-4dca-b57b-5c8ae716d35b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:56.048632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0618d54-c1c4-4dca-b57b-5c8ae716d35b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:56.048908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0618d54-c1c4-4dca-b57b-5c8ae716d35b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:56.065414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0618d54-c1c4-4dca-b57b-5c8ae716d35b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:56.065414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0618d54-c1c4-4dca-b57b-5c8ae716d35b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:56.082663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-e0618d54-c1c4-4dca-b57b-5c8ae716d35b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:56.082663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:56.082663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:56.082663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:56.082663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:56.082663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:56.082663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:56.082663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:56.082663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:56.082663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:56.082663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:56.082663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:56.082663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:56.082663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:56.082663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:56.083633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:56.083633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:56.083633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:56.083633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:56.083633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:56.083633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:56.083633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:56.083633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:56.083633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:56.083633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:56.083633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:56.083633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:56.083633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:56.083633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:56.083633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:56.083633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:56.083633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:56.084554 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:56.084554 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:56.084554 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:56.084554 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:56.087553 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 419/864] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:56.092910 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cee394-70f8-42b1-a940-775d61390abe None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:56.213327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cee394-70f8-42b1-a940-775d61390abe None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:56.250554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e092f-f799-426f-b4b1-caccfd740aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:56.251177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e092f-f799-426f-b4b1-caccfd740aaa None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:56.251466 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e092f-f799-426f-b4b1-caccfd740aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:56.251744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e092f-f799-426f-b4b1-caccfd740aaa None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:56.252442 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e092f-f799-426f-b4b1-caccfd740aaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:56.252753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e092f-f799-426f-b4b1-caccfd740aaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:56.254160 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e092f-f799-426f-b4b1-caccfd740aaa None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:56.267378 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 446/865] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:56.272894 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0618d54-c1c4-4dca-b57b-5c8ae716d35b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:56.323775 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:56.382802 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0618d54-c1c4-4dca-b57b-5c8ae716d35b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:56.418618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32eaa1-5b3f-4ae9-ab91-bf592183b63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:56.419613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32eaa1-5b3f-4ae9-ab91-bf592183b63d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:56.420113 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32eaa1-5b3f-4ae9-ab91-bf592183b63d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:56.420592 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32eaa1-5b3f-4ae9-ab91-bf592183b63d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:56.428896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea32eaa1-5b3f-4ae9-ab91-bf592183b63d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:56.430643 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea32eaa1-5b3f-4ae9-ab91-bf592183b63d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:56.442628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ea32eaa1-5b3f-4ae9-ab91-bf592183b63d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 22:15:56.442628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:56.442628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:56.442628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:56.442628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:56.442628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:56.442628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:56.442628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:56.442628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:56.442628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:56.442628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:56.442628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:56.442628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:15:56.442628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:15:56.442628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:15:56.442628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:15:56.443983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:56.443983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:56.443983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:15:56.443983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:56.443983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:15:56.443983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:15:56.443983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:56.443983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:56.443983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:56.443983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:56.443983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:56.443983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:56.443983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:56.443983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:56.443983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:56.443983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:56.443983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:56.443983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:56.444941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:56.444941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:56.444941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:15:56.444941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:15:56.444941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:15:56.444941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:15:56.444941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:15:56.444941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:15:56.444941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 22:15:56.444941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:56.447688 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 420/866] 10.222.0.39 () {56 vars in 1117 bytes} [Thu Jul 28 22:15:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:56.453561 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e092f-f799-426f-b4b1-caccfd740aaa None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:56.482264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:56.564904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e092f-f799-426f-b4b1-caccfd740aaa None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:56.603661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3892e4-16ac-4229-8cfa-42636356d5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:56.604306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3892e4-16ac-4229-8cfa-42636356d5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:56.604661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3892e4-16ac-4229-8cfa-42636356d5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:56.604948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3892e4-16ac-4229-8cfa-42636356d5e2 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:56.605790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3892e4-16ac-4229-8cfa-42636356d5e2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:56.606110 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3892e4-16ac-4229-8cfa-42636356d5e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:56.607570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3892e4-16ac-4229-8cfa-42636356d5e2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:56.627692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 447/867] 10.222.0.39 () {56 vars in 1149 bytes} [Thu Jul 28 22:15:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:56.633364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32eaa1-5b3f-4ae9-ab91-bf592183b63d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:56.740834 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32eaa1-5b3f-4ae9-ab91-bf592183b63d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:56.748269 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:56.778777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b774ec-7500-485c-a138-b17c3c1ddadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:56.779403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b774ec-7500-485c-a138-b17c3c1ddadf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:56.779690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b774ec-7500-485c-a138-b17c3c1ddadf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:56.779965 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b774ec-7500-485c-a138-b17c3c1ddadf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:56.854097 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b774ec-7500-485c-a138-b17c3c1ddadf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:56.855562 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b774ec-7500-485c-a138-b17c3c1ddadf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:56.959324 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 421/868] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:15:56 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 22:15:56.965448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3892e4-16ac-4229-8cfa-42636356d5e2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:57.023195 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db3984-1d73-42e9-8ebe-010d0bb252c1 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:57.024157 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db3984-1d73-42e9-8ebe-010d0bb252c1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:57.024440 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db3984-1d73-42e9-8ebe-010d0bb252c1 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:57.026414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 422/869] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:57.036130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fd7c5-4fd2-482e-b38e-4a8812914a2c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:57.036416 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fd7c5-4fd2-482e-b38e-4a8812914a2c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:57.036695 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926fd7c5-4fd2-482e-b38e-4a8812914a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:57.050140 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:57.064641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:57.065371 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-926fd7c5-4fd2-482e-b38e-4a8812914a2c None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:57.067071 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-926fd7c5-4fd2-482e-b38e-4a8812914a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:57.087758 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3892e4-16ac-4229-8cfa-42636356d5e2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:57.125492 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e15ed-33b3-41c6-85f7-f60fe04a5212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:57.126443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e15ed-33b3-41c6-85f7-f60fe04a5212 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:57.126926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e15ed-33b3-41c6-85f7-f60fe04a5212 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:57.127372 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e15ed-33b3-41c6-85f7-f60fe04a5212 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:57.138361 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e15ed-33b3-41c6-85f7-f60fe04a5212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:57.140308 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e15ed-33b3-41c6-85f7-f60fe04a5212 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:57.153552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 423/870] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:57 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:15:57.153552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871c51b-0f6f-4aa8-aff9-bc34b84f0104 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:57.153552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871c51b-0f6f-4aa8-aff9-bc34b84f0104 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:57.153552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871c51b-0f6f-4aa8-aff9-bc34b84f0104 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:57.153552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 424/871] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:57.158782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-329e15ed-33b3-41c6-85f7-f60fe04a5212 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:57.158782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:57.158782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:57.158782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:57.158782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:57.158782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:57.158782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:57.158782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:57.158782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:57.158782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:57.158782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:57.158782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:57.158782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:15:57.158782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:15:57.158782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:15:57.159643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:15:57.159643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:57.159643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:57.159643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:57.159643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:57.159643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:57.159643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:57.159643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:57.159643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:57.159643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:57.159643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:57.159643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:57.159643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:57.159643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:57.159643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:57.159643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:57.159643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:57.160554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:15:57.160554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:15:57.160554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:15:57.160554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:57.162025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 448/872] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:15:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:57.163646 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0566fbe-a696-4c31-ab03-e9dc1e0f95ef None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:57.165564 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0566fbe-a696-4c31-ab03-e9dc1e0f95ef None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:57.165564 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0566fbe-a696-4c31-ab03-e9dc1e0f95ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:57.166435 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e15ed-33b3-41c6-85f7-f60fe04a5212 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:57.183998 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e0566fbe-a696-4c31-ab03-e9dc1e0f95ef None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:57.185196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e0566fbe-a696-4c31-ab03-e9dc1e0f95ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:57.277744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 425/873] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:57 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:15:57.278786 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0566fbe-a696-4c31-ab03-e9dc1e0f95ef None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:57.307655 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e15ed-33b3-41c6-85f7-f60fe04a5212 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:57.352353 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d22d4-89ce-49c8-a57c-6c01628fe762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:57.353701 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d22d4-89ce-49c8-a57c-6c01628fe762 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:57.354142 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d22d4-89ce-49c8-a57c-6c01628fe762 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:57.354569 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d22d4-89ce-49c8-a57c-6c01628fe762 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:57.355828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d22d4-89ce-49c8-a57c-6c01628fe762 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:57.356299 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d22d4-89ce-49c8-a57c-6c01628fe762 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:57.357873 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0d22d4-89ce-49c8-a57c-6c01628fe762 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:57.359501 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:57.361506 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:57.373477 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 449/874] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:57.374418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d22d4-89ce-49c8-a57c-6c01628fe762 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:57.402158 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0566fbe-a696-4c31-ab03-e9dc1e0f95ef None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:57.439114 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef0298-9b6e-454b-9977-aa338f049fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:57.440060 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef0298-9b6e-454b-9977-aa338f049fcd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:57.440346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef0298-9b6e-454b-9977-aa338f049fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:57.440622 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef0298-9b6e-454b-9977-aa338f049fcd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:57.441356 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef0298-9b6e-454b-9977-aa338f049fcd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:57.442668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef0298-9b6e-454b-9977-aa338f049fcd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:57.453388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-eeef0298-9b6e-454b-9977-aa338f049fcd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 22:15:57.453388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:57.453388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:15:57.453388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:15:57.453388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:15:57.453388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:15:57.453388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:15:57.453388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:57.453388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 22:15:57.453388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:57.453388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:15:57.453388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:57.453388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:57.453388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:57.453388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:57.453388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:57.453388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:57.454644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:57.454644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:57.454644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:57.454644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:57.454644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:57.454644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:57.454644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:15:57.454644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:15:57.454644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:15:57.454644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:15:57.454644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:57.454644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:57.454644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:57.454644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:57.454644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:57.454644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:57.458613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:57.458613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:57.458613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:57.458613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:57.458613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:57.458613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:57.458613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:57.458613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:57.458613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:57.458613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:57.458613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:15:57.458613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:57.458613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 22:15:57.458613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:57.458613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 426/875] 10.222.0.39 () {56 vars in 1126 bytes} [Thu Jul 28 22:15:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:57.459995 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6103d-9d12-4d59-b02f-65fbbb1c17f2 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:57.459995 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6103d-9d12-4d59-b02f-65fbbb1c17f2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:57.459995 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f6103d-9d12-4d59-b02f-65fbbb1c17f2 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:57.459995 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 427/876] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:57.459995 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6103d-9d12-4d59-b02f-65fbbb1c17f2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:57.485207 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0d22d4-89ce-49c8-a57c-6c01628fe762 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:57.525282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f8769-af66-43ae-aec5-472a8371840e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:57.526206 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f8769-af66-43ae-aec5-472a8371840e None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:57.526701 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f8769-af66-43ae-aec5-472a8371840e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:57.527089 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f8769-af66-43ae-aec5-472a8371840e None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:57.527857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f8769-af66-43ae-aec5-472a8371840e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:57.529218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5f8769-af66-43ae-aec5-472a8371840e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:57.557645 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 450/877] 10.222.0.39 () {60 vars in 1141 bytes} [Thu Jul 28 22:15:57 2022] POST /identity/v3/services => generated 269 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:15:57.557645 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f8769-af66-43ae-aec5-472a8371840e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:57.594961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f6103d-9d12-4d59-b02f-65fbbb1c17f2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:57.627665 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:57.634345 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:57.640766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e518f7b-b90d-4c8a-8d9a-c607afab8d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:57.641552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e518f7b-b90d-4c8a-8d9a-c607afab8d01 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:57.641894 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e518f7b-b90d-4c8a-8d9a-c607afab8d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:57.642242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e518f7b-b90d-4c8a-8d9a-c607afab8d01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:57.650459 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e518f7b-b90d-4c8a-8d9a-c607afab8d01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:57.651957 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e518f7b-b90d-4c8a-8d9a-c607afab8d01 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:57.665316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-2e518f7b-b90d-4c8a-8d9a-c607afab8d01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:57.665316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:57.665316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:57.665316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:57.665316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:57.665316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:57.665316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:57.665316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:57.665316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:57.665316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:57.665316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:57.665316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:57.665316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:15:57.665316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:15:57.665316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:15:57.666464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:15:57.666464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:57.666464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:57.666464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:57.666464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:57.666464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:57.666464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:57.666464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:57.666464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:57.666464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:57.666464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:57.666464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:57.666464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:57.666464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:57.666464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:57.666464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:57.666464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:57.667901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:15:57.667901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:15:57.667901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:15:57.667901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:15:57.667901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:15:57.667901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:15:57.667901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:15:57.667901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:57.667901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 428/878] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:15:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:57.668394 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66982e86-7429-435a-a746-b89a867750ad None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:57.668394 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66982e86-7429-435a-a746-b89a867750ad None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:57.668746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66982e86-7429-435a-a746-b89a867750ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:57.688903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-66982e86-7429-435a-a746-b89a867750ad None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:57.690210 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-66982e86-7429-435a-a746-b89a867750ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:57.691199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f8769-af66-43ae-aec5-472a8371840e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:57.732085 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a5444-6de7-40ea-bc5e-97d9cc3d391b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:57.732717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a5444-6de7-40ea-bc5e-97d9cc3d391b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:57.733186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a5444-6de7-40ea-bc5e-97d9cc3d391b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:57.733469 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a5444-6de7-40ea-bc5e-97d9cc3d391b None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:57.735078 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a5444-6de7-40ea-bc5e-97d9cc3d391b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:57.735389 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a5444-6de7-40ea-bc5e-97d9cc3d391b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:57.736766 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8a5444-6de7-40ea-bc5e-97d9cc3d391b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:57.742229 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 451/879] 10.222.0.39 () {56 vars in 1122 bytes} [Thu Jul 28 22:15:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:57.743113 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a5444-6de7-40ea-bc5e-97d9cc3d391b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:57.770474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 429/880] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:57 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:15:57.771381 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66982e86-7429-435a-a746-b89a867750ad None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:57.864969 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a5444-6de7-40ea-bc5e-97d9cc3d391b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:57.877001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66982e86-7429-435a-a746-b89a867750ad None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:57.884055 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:57.894882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:57.903846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a7603-45d6-4351-b713-9deca414a4d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:57.904470 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a7603-45d6-4351-b713-9deca414a4d6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:57.904751 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a7603-45d6-4351-b713-9deca414a4d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:57.905022 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77a7603-45d6-4351-b713-9deca414a4d6 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:57.905796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a7603-45d6-4351-b713-9deca414a4d6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:57.906103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a7603-45d6-4351-b713-9deca414a4d6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:57.907625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77a7603-45d6-4351-b713-9deca414a4d6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:57.916310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f699854-f481-4467-84ef-7a9aa3422c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:57.916868 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f699854-f481-4467-84ef-7a9aa3422c47 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:57.917151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f699854-f481-4467-84ef-7a9aa3422c47 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:57.917429 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f699854-f481-4467-84ef-7a9aa3422c47 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:57.918147 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f699854-f481-4467-84ef-7a9aa3422c47 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:57.918484 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f699854-f481-4467-84ef-7a9aa3422c47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:57.919827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f699854-f481-4467-84ef-7a9aa3422c47 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:57.921093 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 452/881] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:15:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:57.921847 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a7603-45d6-4351-b713-9deca414a4d6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:57.925444 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 430/882] 10.222.0.39 () {56 vars in 1122 bytes} [Thu Jul 28 22:15:57 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:57.926759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f699854-f481-4467-84ef-7a9aa3422c47 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:58.052728 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f699854-f481-4467-84ef-7a9aa3422c47 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:58.070605 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77a7603-45d6-4351-b713-9deca414a4d6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:58.098277 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a276c8a-1255-4dd6-94f5-011312149ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:58.098895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a276c8a-1255-4dd6-94f5-011312149ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:58.099173 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a276c8a-1255-4dd6-94f5-011312149ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:58.099441 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a276c8a-1255-4dd6-94f5-011312149ed6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:58.100228 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a276c8a-1255-4dd6-94f5-011312149ed6 None admin] RBAC: Adding query filter params (role.id=a66ab77a63174592a1e468d2464e8bc4, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=85626350279c40ea99da93b9ed6994b2) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:58.100534 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a276c8a-1255-4dd6-94f5-011312149ed6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:58.103087 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a276c8a-1255-4dd6-94f5-011312149ed6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:58.108614 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 431/883] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:15:57 2022] GET /identity/v3/role_assignments?user.id=85626350279c40ea99da93b9ed6994b2&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=a66ab77a63174592a1e468d2464e8bc4 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:15:58.126396 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669a971-e76a-4c34-9b57-2b446dcf1c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:58.126617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669a971-e76a-4c34-9b57-2b446dcf1c45 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:58.126919 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669a971-e76a-4c34-9b57-2b446dcf1c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:58.127028 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3669a971-e76a-4c34-9b57-2b446dcf1c45 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:58.127666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3669a971-e76a-4c34-9b57-2b446dcf1c45 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:58.129094 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3669a971-e76a-4c34-9b57-2b446dcf1c45 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:58.143420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-3669a971-e76a-4c34-9b57-2b446dcf1c45 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 22:15:58.143420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:58.143420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:15:58.143420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:15:58.143420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:15:58.143420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:15:58.143420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:15:58.143420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:58.143420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 22:15:58.143420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:58.143420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:15:58.143420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:58.143420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:58.143420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:58.143420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:58.143420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:58.143420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:58.144493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:58.144493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:58.144493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:58.144493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:58.144493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:58.144493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:58.144493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:15:58.144493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:15:58.144493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:15:58.144493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:15:58.144493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:58.144493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:58.144493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:58.144493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:58.144493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:58.144493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:58.145803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:58.145803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:58.145803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:58.145803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:58.145803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:58.145803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:58.145803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:58.145803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:58.145803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:58.145803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:58.145803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:15:58.145803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:58.145803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 22:15:58.145803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:15:58.145803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 453/884] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:15:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:58.156581 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a276c8a-1255-4dd6-94f5-011312149ed6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:58.209134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:58.275188 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a276c8a-1255-4dd6-94f5-011312149ed6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:58.312796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c58c8-887e-4734-b03e-2e5d41d4c5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:58.313414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c58c8-887e-4734-b03e-2e5d41d4c5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:58.313723 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c58c8-887e-4734-b03e-2e5d41d4c5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:58.313996 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c58c8-887e-4734-b03e-2e5d41d4c5b7 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:58.314682 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c58c8-887e-4734-b03e-2e5d41d4c5b7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:58.314981 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c58c8-887e-4734-b03e-2e5d41d4c5b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:58.316411 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638c58c8-887e-4734-b03e-2e5d41d4c5b7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:58.325604 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 432/885] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:15:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:58.327417 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669a971-e76a-4c34-9b57-2b446dcf1c45 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:58.377464 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:58.439187 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3669a971-e76a-4c34-9b57-2b446dcf1c45 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:58.475252 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4447bf-557a-4df3-9122-79bc2b3f5593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:58.476208 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4447bf-557a-4df3-9122-79bc2b3f5593 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:58.476735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4447bf-557a-4df3-9122-79bc2b3f5593 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:58.477371 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4447bf-557a-4df3-9122-79bc2b3f5593 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:58.478280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4447bf-557a-4df3-9122-79bc2b3f5593 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:58.478805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4447bf-557a-4df3-9122-79bc2b3f5593 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:58.480373 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4447bf-557a-4df3-9122-79bc2b3f5593 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:58.486120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 454/886] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:15:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:58.494622 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c58c8-887e-4734-b03e-2e5d41d4c5b7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:58.603624 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c58c8-887e-4734-b03e-2e5d41d4c5b7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:58.613411 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:58.641876 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48461a17-33ab-441d-83d7-7a67d153dcb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:58.642845 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48461a17-33ab-441d-83d7-7a67d153dcb4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:58.643355 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48461a17-33ab-441d-83d7-7a67d153dcb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:58.643847 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48461a17-33ab-441d-83d7-7a67d153dcb4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:58.644807 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48461a17-33ab-441d-83d7-7a67d153dcb4 None admin] RBAC: Adding query filter params (interface=public, service_id=2e3132a93b3a49c485574aa2d733eb28, region_id=RegionOne) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:15:58.645317 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48461a17-33ab-441d-83d7-7a67d153dcb4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:58.646909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48461a17-33ab-441d-83d7-7a67d153dcb4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:58.653141 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 433/887] 10.222.0.39 () {56 vars in 1251 bytes} [Thu Jul 28 22:15:58 2022] GET /identity/v3/endpoints?service_id=2e3132a93b3a49c485574aa2d733eb28&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:15:58.664822 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4447bf-557a-4df3-9122-79bc2b3f5593 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:58.772461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4447bf-557a-4df3-9122-79bc2b3f5593 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:58.780491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:15:58.812745 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f92b92-15c6-4288-8996-9d37befda095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:58.813369 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f92b92-15c6-4288-8996-9d37befda095 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:58.813683 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f92b92-15c6-4288-8996-9d37befda095 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:58.813962 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f92b92-15c6-4288-8996-9d37befda095 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:58.814619 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f92b92-15c6-4288-8996-9d37befda095 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:58.815987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f92b92-15c6-4288-8996-9d37befda095 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:58.822685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 455/888] 10.222.0.39 () {56 vars in 1087 bytes} [Thu Jul 28 22:15:58 2022] GET /identity/v3/services => generated 1268 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:15:59.635915 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721b230-7273-4970-a202-38222f0b0f32 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:59.636438 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721b230-7273-4970-a202-38222f0b0f32 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:59.636516 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721b230-7273-4970-a202-38222f0b0f32 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:59.638112 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 434/889] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:15:59.643458 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602cb7c-e6cb-4ded-8fd8-8f12a7072416 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:59.643846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602cb7c-e6cb-4ded-8fd8-8f12a7072416 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:59.644021 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602cb7c-e6cb-4ded-8fd8-8f12a7072416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:59.662621 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-4602cb7c-e6cb-4ded-8fd8-8f12a7072416 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:15:59.663849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4602cb7c-e6cb-4ded-8fd8-8f12a7072416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:15:59.736561 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 456/890] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:15:59 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:15:59.758906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721b230-7273-4970-a202-38222f0b0f32 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:15:59.870633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721b230-7273-4970-a202-38222f0b0f32 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:15:59.907670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5e1dc-bfad-4943-a484-ebe59a0d434d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:15:59.908244 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5e1dc-bfad-4943-a484-ebe59a0d434d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:15:59.908526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5e1dc-bfad-4943-a484-ebe59a0d434d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:15:59.908803 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5e1dc-bfad-4943-a484-ebe59a0d434d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:15:59.909436 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5e1dc-bfad-4943-a484-ebe59a0d434d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:15:59.910832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5e1dc-bfad-4943-a484-ebe59a0d434d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:15:59.920196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-3fb5e1dc-bfad-4943-a484-ebe59a0d434d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 22:15:59.920196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:59.920196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:15:59.920196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:15:59.920196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:15:59.920196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:15:59.920196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:15:59.920196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:59.920196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 22:15:59.920196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:59.920196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:15:59.920196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:59.920196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:15:59.920196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:15:59.920196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:15:59.920196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:15:59.921164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:15:59.921164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:15:59.921164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:15:59.921164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:15:59.921164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:15:59.921164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:15:59.921164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:15:59.921164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:15:59.921164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:15:59.921164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:15:59.921164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:15:59.921164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:15:59.921164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:15:59.921164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:15:59.921164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:15:59.921164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:15:59.922205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:15:59.922205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:15:59.922205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:15:59.922205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:15:59.922205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:15:59.922205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:15:59.922205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:15:59.922205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:15:59.922205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:15:59.922205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:15:59.922205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:15:59.922205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:15:59.922205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:15:59.922205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 22:15:59.922205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:15:59.922205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 435/891] 10.222.0.39 () {56 vars in 1132 bytes} [Thu Jul 28 22:15:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:15:59.931127 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602cb7c-e6cb-4ded-8fd8-8f12a7072416 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:00.014287 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253518b-ca7f-40e0-bbd5-bdde9109bb5f None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:00.014654 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253518b-ca7f-40e0-bbd5-bdde9109bb5f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:00.016173 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253518b-ca7f-40e0-bbd5-bdde9109bb5f None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:00.017833 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 436/892] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:16:00.022871 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bef1d-3967-40df-a46d-e7714742aa46 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:00.023682 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bef1d-3967-40df-a46d-e7714742aa46 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:00.023962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bef1d-3967-40df-a46d-e7714742aa46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:00.036853 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:00.041567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602cb7c-e6cb-4ded-8fd8-8f12a7072416 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:00.045990 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0a1bef1d-3967-40df-a46d-e7714742aa46 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:00.047203 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0a1bef1d-3967-40df-a46d-e7714742aa46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:00.080003 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:00.088029 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ba770-b4ea-4e71-ba04-0f7d0b62a450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:00.088659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ba770-b4ea-4e71-ba04-0f7d0b62a450 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:00.088945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ba770-b4ea-4e71-ba04-0f7d0b62a450 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:00.089223 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ba770-b4ea-4e71-ba04-0f7d0b62a450 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:00.090124 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ba770-b4ea-4e71-ba04-0f7d0b62a450 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:00.090487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ba770-b4ea-4e71-ba04-0f7d0b62a450 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:00.092492 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ba770-b4ea-4e71-ba04-0f7d0b62a450 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:00.096893 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 457/893] 10.222.0.39 () {56 vars in 1126 bytes} [Thu Jul 28 22:15:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:00.103207 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ba770-b4ea-4e71-ba04-0f7d0b62a450 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:00.133058 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 437/894] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:16:00.157561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bef1d-3967-40df-a46d-e7714742aa46 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:00.221606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ba770-b4ea-4e71-ba04-0f7d0b62a450 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:00.266089 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c00f4-ee60-4f89-a2a3-ddee8a2bd0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:00.266871 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c00f4-ee60-4f89-a2a3-ddee8a2bd0ca None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:00.267244 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c00f4-ee60-4f89-a2a3-ddee8a2bd0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:00.267610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c00f4-ee60-4f89-a2a3-ddee8a2bd0ca None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:00.268400 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c00f4-ee60-4f89-a2a3-ddee8a2bd0ca None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:00.268800 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c00f4-ee60-4f89-a2a3-ddee8a2bd0ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:00.269368 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bef1d-3967-40df-a46d-e7714742aa46 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:00.270420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c00f4-ee60-4f89-a2a3-ddee8a2bd0ca None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:00.281475 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 458/895] 10.222.0.39 () {56 vars in 1126 bytes} [Thu Jul 28 22:16:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:00.282443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c00f4-ee60-4f89-a2a3-ddee8a2bd0ca None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:00.306825 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49363c54-7e74-4c07-839b-7182a842f2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:00.307790 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49363c54-7e74-4c07-839b-7182a842f2d2 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:00.308236 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49363c54-7e74-4c07-839b-7182a842f2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:00.308654 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49363c54-7e74-4c07-839b-7182a842f2d2 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:00.309486 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49363c54-7e74-4c07-839b-7182a842f2d2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:00.311234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49363c54-7e74-4c07-839b-7182a842f2d2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:00.314308 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:00.332687 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 438/896] 10.222.0.39 () {60 vars in 1141 bytes} [Thu Jul 28 22:16:00 2022] POST /identity/v3/services => generated 241 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:16:00.386489 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be8188-9ff4-44ea-82ca-5e32c58bd932 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:00.387535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be8188-9ff4-44ea-82ca-5e32c58bd932 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:00.388027 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57be8188-9ff4-44ea-82ca-5e32c58bd932 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:00.390315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 439/897] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:00.396710 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf585301-61f7-4841-8598-64b25df7ff5a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:00.397372 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf585301-61f7-4841-8598-64b25df7ff5a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:00.397919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf585301-61f7-4841-8598-64b25df7ff5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:00.407288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c00f4-ee60-4f89-a2a3-ddee8a2bd0ca None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:00.419086 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-cf585301-61f7-4841-8598-64b25df7ff5a None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:00.420767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cf585301-61f7-4841-8598-64b25df7ff5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:00.444324 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:00.460244 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fadb4-dfcf-4d79-93ee-db1a2ae3c5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:00.461033 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fadb4-dfcf-4d79-93ee-db1a2ae3c5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:00.461312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fadb4-dfcf-4d79-93ee-db1a2ae3c5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:00.461611 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011fadb4-dfcf-4d79-93ee-db1a2ae3c5f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:00.462384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011fadb4-dfcf-4d79-93ee-db1a2ae3c5f3 None admin] RBAC: Adding query filter params (interface=public, service_id=23cc85e4b2e5401f9e81e79da3b99d7c, region_id=RegionOne) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:00.462694 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011fadb4-dfcf-4d79-93ee-db1a2ae3c5f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:00.464151 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011fadb4-dfcf-4d79-93ee-db1a2ae3c5f3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:00.473516 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 459/898] 10.222.0.39 () {56 vars in 1251 bytes} [Thu Jul 28 22:16:00 2022] GET /identity/v3/endpoints?service_id=23cc85e4b2e5401f9e81e79da3b99d7c&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:00.479968 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fadb4-dfcf-4d79-93ee-db1a2ae3c5f3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:00.503632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 440/899] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:16:00.528723 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf585301-61f7-4841-8598-64b25df7ff5a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:00.583421 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:00.598068 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011fadb4-dfcf-4d79-93ee-db1a2ae3c5f3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:00.644743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eea556-355e-41ae-972c-88263b3cb0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:00.645475 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eea556-355e-41ae-972c-88263b3cb0fd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:00.645731 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eea556-355e-41ae-972c-88263b3cb0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:00.645961 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eea556-355e-41ae-972c-88263b3cb0fd None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:00.646638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eea556-355e-41ae-972c-88263b3cb0fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:00.649215 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eea556-355e-41ae-972c-88263b3cb0fd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:00.658124 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 460/900] 10.222.0.39 () {56 vars in 1087 bytes} [Thu Jul 28 22:16:00 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:16:00.680312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf585301-61f7-4841-8598-64b25df7ff5a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:00.722609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6d90d-d9cd-4779-8a5b-e55cbdddcbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:00.723710 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6d90d-d9cd-4779-8a5b-e55cbdddcbc0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:00.724180 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6d90d-d9cd-4779-8a5b-e55cbdddcbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:00.724687 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6d90d-d9cd-4779-8a5b-e55cbdddcbc0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:00.738539 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6d90d-d9cd-4779-8a5b-e55cbdddcbc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:16:00.744401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-e0e6d90d-d9cd-4779-8a5b-e55cbdddcbc0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 22:16:00.744401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:00.744401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:00.744401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:00.744401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:00.744401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:00.744401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:00.744401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:00.744401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:00.744401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:00.744401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:00.744401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:00.744401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:16:00.744401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:16:00.744401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:16:00.744401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:16:00.745427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:16:00.745427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:16:00.745427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:16:00.745427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:16:00.745427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:00.745427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:00.745427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:00.745427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:00.745427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:00.745427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:00.745427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:00.745427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:00.745427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:00.745427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:00.745427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:00.745427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:00.746575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:00.746575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:00.746575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:00.746575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:00.746575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:16:00.746575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:16:00.746575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:16:00.746575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:16:00.746575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:16:00.746575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:16:00.746575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 22:16:00.746575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:00.749102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 441/901] 10.222.0.39 () {56 vars in 1120 bytes} [Thu Jul 28 22:16:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:00.755289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eea556-355e-41ae-972c-88263b3cb0fd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:00.882301 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eea556-355e-41ae-972c-88263b3cb0fd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:00.911322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3787c-b895-4e93-a7dc-023bceef4c2d None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:00.911600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3787c-b895-4e93-a7dc-023bceef4c2d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:00.911875 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3787c-b895-4e93-a7dc-023bceef4c2d None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:00.913455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 442/902] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:16:00.919594 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d464f5-21d4-4b70-ac80-34a9dff817cd None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:00.919881 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d464f5-21d4-4b70-ac80-34a9dff817cd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:00.920156 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d464f5-21d4-4b70-ac80-34a9dff817cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:00.927951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81e37f-bfd7-4c79-9ca0-25e4358365b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:00.928884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81e37f-bfd7-4c79-9ca0-25e4358365b4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:00.929351 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81e37f-bfd7-4c79-9ca0-25e4358365b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:00.929834 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81e37f-bfd7-4c79-9ca0-25e4358365b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:00.930746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81e37f-bfd7-4c79-9ca0-25e4358365b4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:00.931233 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81e37f-bfd7-4c79-9ca0-25e4358365b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:00.932893 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81e37f-bfd7-4c79-9ca0-25e4358365b4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:00.941994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 461/903] 10.222.0.39 () {56 vars in 1115 bytes} [Thu Jul 28 22:16:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:00.945677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-08d464f5-21d4-4b70-ac80-34a9dff817cd None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:00.945677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-08d464f5-21d4-4b70-ac80-34a9dff817cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:00.948480 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81e37f-bfd7-4c79-9ca0-25e4358365b4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:00.962915 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:00.971084 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:01.050715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 443/904] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:01.075900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d464f5-21d4-4b70-ac80-34a9dff817cd None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:01.082549 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81e37f-bfd7-4c79-9ca0-25e4358365b4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:01.133868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881198d0-561d-4819-ad6b-20e3798c8457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:01.135227 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881198d0-561d-4819-ad6b-20e3798c8457 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:01.135809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881198d0-561d-4819-ad6b-20e3798c8457 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:01.136347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881198d0-561d-4819-ad6b-20e3798c8457 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:01.152358 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881198d0-561d-4819-ad6b-20e3798c8457 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:01.157437 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881198d0-561d-4819-ad6b-20e3798c8457 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:01.180698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-881198d0-561d-4819-ad6b-20e3798c8457 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:01.180698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:01.180698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:01.180698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:01.180698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:01.180698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:01.180698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:01.180698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:01.180698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:01.180698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:01.180698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:01.180698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:01.180698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:16:01.180698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:16:01.180698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:16:01.181891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:16:01.181891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:01.181891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:01.181891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:01.181891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:01.181891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:01.181891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:01.181891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:01.181891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:01.181891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:01.181891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:01.181891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:01.181891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:01.181891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:01.181891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:01.181891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:01.181891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:01.182860 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:16:01.182860 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:16:01.182860 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:01.182860 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:01.188396 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 462/905] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:01.189405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881198d0-561d-4819-ad6b-20e3798c8457 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:01.203722 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d464f5-21d4-4b70-ac80-34a9dff817cd None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:01.256811 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b503b-8512-4300-8067-8a09a1e8f2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:01.259095 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b503b-8512-4300-8067-8a09a1e8f2b6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:01.259584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b503b-8512-4300-8067-8a09a1e8f2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:01.260077 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b503b-8512-4300-8067-8a09a1e8f2b6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:01.260891 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b503b-8512-4300-8067-8a09a1e8f2b6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:01.263333 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b503b-8512-4300-8067-8a09a1e8f2b6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:01.276759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-410b503b-8512-4300-8067-8a09a1e8f2b6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 22:16:01.276759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:01.276759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:16:01.276759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:16:01.276759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:16:01.276759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:16:01.276759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:16:01.276759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:01.276759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 22:16:01.276759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:01.276759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:16:01.276759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:01.276759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:01.276759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:01.276759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:01.276759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:01.276759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:01.277844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:01.277844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:01.277844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:01.277844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:01.277844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:01.277844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:01.277844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:16:01.277844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:16:01.277844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:16:01.277844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:16:01.277844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:01.277844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:01.277844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:01.277844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:01.277844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:01.277844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:01.278679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:01.278679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:01.278679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:01.278679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:01.278679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:01.278679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:01.278679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:01.278679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:01.278679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:01.278679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:01.278679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:16:01.278679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:01.278679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 28 22:16:01.278679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:01.279950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:01.288518 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 444/906] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:16:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:01.291914 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b503b-8512-4300-8067-8a09a1e8f2b6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:01.308831 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:01.338452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881198d0-561d-4819-ad6b-20e3798c8457 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:01.385267 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ddcb2-761a-4bf6-af05-53047a27b502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:01.386225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ddcb2-761a-4bf6-af05-53047a27b502 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:01.386519 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ddcb2-761a-4bf6-af05-53047a27b502 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:01.386800 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ddcb2-761a-4bf6-af05-53047a27b502 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:01.388521 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ddcb2-761a-4bf6-af05-53047a27b502 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:01.389092 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ddcb2-761a-4bf6-af05-53047a27b502 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:01.390568 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ddcb2-761a-4bf6-af05-53047a27b502 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:01.412547 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 463/907] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:01.413641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ddcb2-761a-4bf6-af05-53047a27b502 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:01.428626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b503b-8512-4300-8067-8a09a1e8f2b6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:01.473509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e8d9a-9133-4bb3-a11a-cef39325bfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:01.474962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e8d9a-9133-4bb3-a11a-cef39325bfb6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:01.475493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e8d9a-9133-4bb3-a11a-cef39325bfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:01.475933 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2e8d9a-9133-4bb3-a11a-cef39325bfb6 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:01.477198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e8d9a-9133-4bb3-a11a-cef39325bfb6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:01.477966 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e8d9a-9133-4bb3-a11a-cef39325bfb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:01.480203 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2e8d9a-9133-4bb3-a11a-cef39325bfb6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:01.488205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 445/908] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:16:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:01.494433 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e8d9a-9133-4bb3-a11a-cef39325bfb6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:01.544019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ddcb2-761a-4bf6-af05-53047a27b502 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:01.573711 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:01.589017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2020b-2973-43ea-a873-4e60b7661509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:01.590185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2020b-2973-43ea-a873-4e60b7661509 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:01.591269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2020b-2973-43ea-a873-4e60b7661509 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:01.591753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2020b-2973-43ea-a873-4e60b7661509 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:01.600461 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:01.605175 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf2020b-2973-43ea-a873-4e60b7661509 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:01.608265 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf2020b-2973-43ea-a873-4e60b7661509 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:01.627298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-9bf2020b-2973-43ea-a873-4e60b7661509 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 22:16:01.627298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:01.627298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:01.627298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:01.627298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:01.627298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:01.627298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:01.627298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:01.627298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:01.627298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:01.627298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:01.627298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:01.627298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:16:01.627298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:16:01.627298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:16:01.627298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:16:01.628349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:01.628349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:01.628349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:16:01.628349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:16:01.628349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:16:01.628349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:16:01.628349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:01.628349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:01.628349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:01.628349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:01.628349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:01.628349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:01.628349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:01.628349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:01.628349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:01.628349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:01.628349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:01.628349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:01.629313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:01.629313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:01.629313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:16:01.629313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:16:01.629313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:16:01.629313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:16:01.629313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:16:01.629313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:16:01.629313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 22:16:01.629313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:01.629874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2e8d9a-9133-4bb3-a11a-cef39325bfb6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:01.638508 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 464/909] 10.222.0.39 () {56 vars in 1117 bytes} [Thu Jul 28 22:16:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:01.641270 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2020b-2973-43ea-a873-4e60b7661509 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:01.669201 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882d20e-5086-43e6-9747-da60fdcafc7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:01.669851 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882d20e-5086-43e6-9747-da60fdcafc7e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:01.670126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882d20e-5086-43e6-9747-da60fdcafc7e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:01.670423 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882d20e-5086-43e6-9747-da60fdcafc7e None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:01.671183 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882d20e-5086-43e6-9747-da60fdcafc7e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:01.671486 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882d20e-5086-43e6-9747-da60fdcafc7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:01.672842 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882d20e-5086-43e6-9747-da60fdcafc7e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:01.683678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 446/910] 10.222.0.39 () {56 vars in 1114 bytes} [Thu Jul 28 22:16:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:01.684789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882d20e-5086-43e6-9747-da60fdcafc7e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:01.764433 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf2020b-2973-43ea-a873-4e60b7661509 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:01.803110 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69445d2-d3db-4f52-8778-456993e5ba9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:01.803730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69445d2-d3db-4f52-8778-456993e5ba9b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:01.804010 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69445d2-d3db-4f52-8778-456993e5ba9b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:01.804346 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69445d2-d3db-4f52-8778-456993e5ba9b None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:01.805147 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69445d2-d3db-4f52-8778-456993e5ba9b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:01.805517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69445d2-d3db-4f52-8778-456993e5ba9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:01.807044 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69445d2-d3db-4f52-8778-456993e5ba9b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:01.825311 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882d20e-5086-43e6-9747-da60fdcafc7e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:01.831448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 465/911] 10.222.0.39 () {56 vars in 1149 bytes} [Thu Jul 28 22:16:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:01.832659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69445d2-d3db-4f52-8778-456993e5ba9b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:01.871845 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5e0fc-0238-4e2f-a897-73da1df0f835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:01.872908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5e0fc-0238-4e2f-a897-73da1df0f835 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:01.873395 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5e0fc-0238-4e2f-a897-73da1df0f835 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:01.873969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5e0fc-0238-4e2f-a897-73da1df0f835 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:01.874888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5e0fc-0238-4e2f-a897-73da1df0f835 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:01.876509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5e0fc-0238-4e2f-a897-73da1df0f835 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:01.882364 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:01.886282 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:01.914079 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 447/912] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:16:01 2022] POST /identity/v3/endpoints => generated 346 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:16:01.958745 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69445d2-d3db-4f52-8778-456993e5ba9b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:02.007602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4a028-f661-4802-baf7-73f2e16fd1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:02.008294 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4a028-f661-4802-baf7-73f2e16fd1fd None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:02.008573 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4a028-f661-4802-baf7-73f2e16fd1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:02.008844 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4a028-f661-4802-baf7-73f2e16fd1fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:02.084004 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4a028-f661-4802-baf7-73f2e16fd1fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:02.085376 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4a028-f661-4802-baf7-73f2e16fd1fd None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:02.126696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:02.188236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 466/913] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:16:01 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 22:16:02.194034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5e0fc-0238-4e2f-a897-73da1df0f835 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:02.305081 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5e0fc-0238-4e2f-a897-73da1df0f835 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:02.342081 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df73a54-6072-482e-a55f-5846e3cd192f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:02.343055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df73a54-6072-482e-a55f-5846e3cd192f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:02.343540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df73a54-6072-482e-a55f-5846e3cd192f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:02.344031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df73a54-6072-482e-a55f-5846e3cd192f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:02.356206 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df73a54-6072-482e-a55f-5846e3cd192f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:02.357976 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df73a54-6072-482e-a55f-5846e3cd192f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:02.373174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7df73a54-6072-482e-a55f-5846e3cd192f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:02.373174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:02.373174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:02.373174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:02.373174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:02.373174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:02.373174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:02.373174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:02.373174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:02.373174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:02.373174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:02.373174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:02.373174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:16:02.373174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:16:02.373174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:16:02.374126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:16:02.374126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:02.374126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:02.374126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:02.374126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:02.374126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:02.374126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:02.374126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:02.374126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:02.374126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:02.374126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:02.374126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:02.374126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:02.374126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:02.374126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:02.374126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:02.374126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:02.375013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:16:02.375013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:16:02.375013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:02.375013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:02.377154 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 448/914] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:02.383411 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4a028-f661-4802-baf7-73f2e16fd1fd None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:02.486442 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4a028-f661-4802-baf7-73f2e16fd1fd None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:02.525853 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b5723d-e014-4900-becb-4d4ae6749a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:02.526477 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5723d-e014-4900-becb-4d4ae6749a00 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:02.526757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5723d-e014-4900-becb-4d4ae6749a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:02.527031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b5723d-e014-4900-becb-4d4ae6749a00 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:02.527726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5723d-e014-4900-becb-4d4ae6749a00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:02.528030 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5723d-e014-4900-becb-4d4ae6749a00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:02.529392 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b5723d-e014-4900-becb-4d4ae6749a00 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:02.541909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:02.542215 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 467/915] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:02.547880 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df73a54-6072-482e-a55f-5846e3cd192f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:02.600474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:02.654174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ff563-27bf-4863-91fd-253a3dc49808 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:02.654542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ff563-27bf-4863-91fd-253a3dc49808 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:02.654765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5ff563-27bf-4863-91fd-253a3dc49808 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:02.656490 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 468/916] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:02.661413 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299dc62-8d8b-4a9f-8b5b-05c1fa0e3404 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:02.661753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299dc62-8d8b-4a9f-8b5b-05c1fa0e3404 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:02.662034 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9299dc62-8d8b-4a9f-8b5b-05c1fa0e3404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:02.666566 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df73a54-6072-482e-a55f-5846e3cd192f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:02.685466 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9299dc62-8d8b-4a9f-8b5b-05c1fa0e3404 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:02.687697 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9299dc62-8d8b-4a9f-8b5b-05c1fa0e3404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:02.707885 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ced46-6da4-442c-b294-9e4d921a0ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:02.709187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ced46-6da4-442c-b294-9e4d921a0ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:02.709700 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ced46-6da4-442c-b294-9e4d921a0ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:02.710152 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025ced46-6da4-442c-b294-9e4d921a0ec1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:02.718787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ced46-6da4-442c-b294-9e4d921a0ec1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:02.720475 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025ced46-6da4-442c-b294-9e4d921a0ec1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:02.733535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-025ced46-6da4-442c-b294-9e4d921a0ec1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:16:02.733535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:02.733535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:02.733535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:02.733535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:02.733535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:02.733535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:02.733535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:02.733535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:02.733535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:02.733535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:02.733535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:02.733535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:16:02.733535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:16:02.733535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:16:02.734526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:16:02.734526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:02.734526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:02.734526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:02.734526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:02.734526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:02.734526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:02.734526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:02.734526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:02.734526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:02.734526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:02.734526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:02.734526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:02.734526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:02.734526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:02.734526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:02.734526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:02.735435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:16:02.735435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:16:02.735435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:16:02.735435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:16:02.735435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:16:02.735435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:16:02.735435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:16:02.735435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:02.735435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 449/917] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:16:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:02.740757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ced46-6da4-442c-b294-9e4d921a0ec1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:02.781753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 469/918] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:16:02.806654 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299dc62-8d8b-4a9f-8b5b-05c1fa0e3404 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:02.879506 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025ced46-6da4-442c-b294-9e4d921a0ec1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:02.890286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:02.923189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9299dc62-8d8b-4a9f-8b5b-05c1fa0e3404 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:02.927344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d09abe2-2417-48d9-bea1-acc49b655b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:02.928207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09abe2-2417-48d9-bea1-acc49b655b49 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:02.928695 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09abe2-2417-48d9-bea1-acc49b655b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:02.929709 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d09abe2-2417-48d9-bea1-acc49b655b49 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:02.930716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09abe2-2417-48d9-bea1-acc49b655b49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:02.931643 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09abe2-2417-48d9-bea1-acc49b655b49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:02.933791 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d09abe2-2417-48d9-bea1-acc49b655b49 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:02.951223 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 450/919] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:16:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:02.952926 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097dd9a-eabe-4938-a28d-55ec662f69f9 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:02.953735 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097dd9a-eabe-4938-a28d-55ec662f69f9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:02.954463 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5097dd9a-eabe-4938-a28d-55ec662f69f9 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:02.955587 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:02.956100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 451/920] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:02.959883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097dd9a-eabe-4938-a28d-55ec662f69f9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:02.961208 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553855c1-43ee-4833-9329-ea79ea937a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:02.961902 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553855c1-43ee-4833-9329-ea79ea937a18 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:02.962191 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553855c1-43ee-4833-9329-ea79ea937a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:02.962468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553855c1-43ee-4833-9329-ea79ea937a18 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:02.963134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553855c1-43ee-4833-9329-ea79ea937a18 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:02.964513 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553855c1-43ee-4833-9329-ea79ea937a18 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:02.973915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-553855c1-43ee-4833-9329-ea79ea937a18 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 22:16:02.973915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:02.973915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:16:02.973915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:16:02.973915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:16:02.973915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:16:02.973915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:16:02.973915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:02.973915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 22:16:02.973915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:02.973915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:16:02.973915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:02.973915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:02.973915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:02.973915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:02.973915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:02.973915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:02.974917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:02.974917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:02.974917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:02.974917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:02.974917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:02.974917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:02.974917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:16:02.974917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:16:02.974917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:16:02.974917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:16:02.974917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:02.974917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:02.974917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:02.974917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:02.974917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:02.974917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:02.975805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:02.975805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:02.975805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:02.975805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:02.975805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:02.975805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:02.975805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:02.975805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:02.975805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:02.975805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:02.975805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:16:02.975805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:02.975805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 22:16:02.975805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:02.975805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 470/921] 10.222.0.39 () {56 vars in 1126 bytes} [Thu Jul 28 22:16:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:02.976656 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1a3ad-69bc-4219-8632-0d23cdec0499 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:02.977283 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1a3ad-69bc-4219-8632-0d23cdec0499 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:02.977283 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb1a3ad-69bc-4219-8632-0d23cdec0499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:02.995796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6fb1a3ad-69bc-4219-8632-0d23cdec0499 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:02.997050 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6fb1a3ad-69bc-4219-8632-0d23cdec0499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:03.075408 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5097dd9a-eabe-4938-a28d-55ec662f69f9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:03.085857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 471/922] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:03.086730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1a3ad-69bc-4219-8632-0d23cdec0499 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:03.119517 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd761f8-c97b-4658-991f-38588a5bfb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:03.120320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd761f8-c97b-4658-991f-38588a5bfb15 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:03.120457 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd761f8-c97b-4658-991f-38588a5bfb15 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:03.120689 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd761f8-c97b-4658-991f-38588a5bfb15 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:03.121484 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd761f8-c97b-4658-991f-38588a5bfb15 None admin] RBAC: Adding query filter params (role.id=62bb6d1764134942a1069bf3a5c1e0c4, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=85626350279c40ea99da93b9ed6994b2) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:03.121815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd761f8-c97b-4658-991f-38588a5bfb15 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:03.123263 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd761f8-c97b-4658-991f-38588a5bfb15 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:03.135831 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 452/923] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:16:02 2022] GET /identity/v3/role_assignments?user.id=85626350279c40ea99da93b9ed6994b2&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=62bb6d1764134942a1069bf3a5c1e0c4 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:03.135831 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd761f8-c97b-4658-991f-38588a5bfb15 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:03.178631 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:03.231261 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb1a3ad-69bc-4219-8632-0d23cdec0499 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:03.259844 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:03.265318 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd761f8-c97b-4658-991f-38588a5bfb15 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:03.272539 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369028d-14c5-4a0d-9b48-736ea69d54ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:03.273805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369028d-14c5-4a0d-9b48-736ea69d54ac None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:03.274223 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369028d-14c5-4a0d-9b48-736ea69d54ac None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:03.274567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369028d-14c5-4a0d-9b48-736ea69d54ac None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:03.275435 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369028d-14c5-4a0d-9b48-736ea69d54ac None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:03.275810 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369028d-14c5-4a0d-9b48-736ea69d54ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:03.277340 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369028d-14c5-4a0d-9b48-736ea69d54ac None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:03.283353 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 472/924] 10.222.0.39 () {56 vars in 1122 bytes} [Thu Jul 28 22:16:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:03.290821 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369028d-14c5-4a0d-9b48-736ea69d54ac None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:03.310611 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9b585-2fe8-473f-9166-26bca03bc841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:03.311424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9b585-2fe8-473f-9166-26bca03bc841 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:03.311811 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9b585-2fe8-473f-9166-26bca03bc841 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:03.312188 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9b585-2fe8-473f-9166-26bca03bc841 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:03.312952 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9b585-2fe8-473f-9166-26bca03bc841 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:03.314721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9b585-2fe8-473f-9166-26bca03bc841 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:03.326289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-0ad9b585-2fe8-473f-9166-26bca03bc841 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 22:16:03.326289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:03.326289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:16:03.326289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:16:03.326289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:16:03.326289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:16:03.326289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:16:03.326289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:03.326289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 22:16:03.326289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:03.326289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:16:03.326289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:03.326289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:03.326289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:03.326289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:03.326289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:03.327522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:03.327522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:03.327522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:03.327522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:03.327522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:03.327522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:03.327522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:03.327522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:16:03.327522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:16:03.327522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:16:03.327522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:16:03.327522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:03.327522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:03.327522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:03.327522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:03.327522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:03.328623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:03.328623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:03.328623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:03.328623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:03.328623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:03.328623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:03.328623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:03.328623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:03.328623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:03.328623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:03.328623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:03.328623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:16:03.328623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:03.328623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 28 22:16:03.328623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:03.330614 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 453/925] 10.222.0.39 () {56 vars in 1132 bytes} [Thu Jul 28 22:16:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:03.339773 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9b585-2fe8-473f-9166-26bca03bc841 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:03.417429 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369028d-14c5-4a0d-9b48-736ea69d54ac None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:03.454370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b22d0e-eacc-4a0c-bc4b-d72c60fb6f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:03.454992 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b22d0e-eacc-4a0c-bc4b-d72c60fb6f58 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:03.455271 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b22d0e-eacc-4a0c-bc4b-d72c60fb6f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:03.455545 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b22d0e-eacc-4a0c-bc4b-d72c60fb6f58 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:03.456458 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b22d0e-eacc-4a0c-bc4b-d72c60fb6f58 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:03.456658 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b22d0e-eacc-4a0c-bc4b-d72c60fb6f58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:03.458120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b22d0e-eacc-4a0c-bc4b-d72c60fb6f58 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:03.464526 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 473/926] 10.222.0.39 () {56 vars in 1122 bytes} [Thu Jul 28 22:16:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:03.473575 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b22d0e-eacc-4a0c-bc4b-d72c60fb6f58 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:03.474377 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:03.483944 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9b585-2fe8-473f-9166-26bca03bc841 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:03.520625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54156c2f-2301-4e0c-b0fd-331aea318efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:03.521442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54156c2f-2301-4e0c-b0fd-331aea318efd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:03.521872 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54156c2f-2301-4e0c-b0fd-331aea318efd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:03.522255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54156c2f-2301-4e0c-b0fd-331aea318efd None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:03.523106 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54156c2f-2301-4e0c-b0fd-331aea318efd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:03.523506 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54156c2f-2301-4e0c-b0fd-331aea318efd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:03.524991 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54156c2f-2301-4e0c-b0fd-331aea318efd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:03.535242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 454/927] 10.222.0.39 () {56 vars in 1126 bytes} [Thu Jul 28 22:16:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:03.536211 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54156c2f-2301-4e0c-b0fd-331aea318efd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:03.542621 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:03.610885 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b22d0e-eacc-4a0c-bc4b-d72c60fb6f58 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:03.651042 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54156c2f-2301-4e0c-b0fd-331aea318efd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:03.659468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0983a-5ee4-41e3-acb2-f46a6c0c096a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:03.660355 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0983a-5ee4-41e3-acb2-f46a6c0c096a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:03.661332 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0983a-5ee4-41e3-acb2-f46a6c0c096a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:03.661789 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0983a-5ee4-41e3-acb2-f46a6c0c096a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:03.662710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0983a-5ee4-41e3-acb2-f46a6c0c096a None admin] RBAC: Adding query filter params (interface=public, service_id=af2dc8b463ba453790500b6ef3914c92, region_id=RegionOne) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:03.663174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0983a-5ee4-41e3-acb2-f46a6c0c096a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:03.666130 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0983a-5ee4-41e3-acb2-f46a6c0c096a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:03.679662 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 474/928] 10.222.0.39 () {56 vars in 1251 bytes} [Thu Jul 28 22:16:03 2022] GET /identity/v3/endpoints?service_id=af2dc8b463ba453790500b6ef3914c92&interface=public®ion_id=RegionOne => generated 194 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:03.680438 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0983a-5ee4-41e3-acb2-f46a6c0c096a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:03.695374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132b742-3830-4833-9be1-91c4bff21537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:03.696073 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132b742-3830-4833-9be1-91c4bff21537 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:03.696318 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132b742-3830-4833-9be1-91c4bff21537 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:03.696567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2132b742-3830-4833-9be1-91c4bff21537 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:03.697307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132b742-3830-4833-9be1-91c4bff21537 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:03.697641 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132b742-3830-4833-9be1-91c4bff21537 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:03.699137 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2132b742-3830-4833-9be1-91c4bff21537 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:03.709065 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 455/929] 10.222.0.39 () {56 vars in 1126 bytes} [Thu Jul 28 22:16:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:03.709972 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132b742-3830-4833-9be1-91c4bff21537 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:03.736498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:03.804331 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0983a-5ee4-41e3-acb2-f46a6c0c096a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:03.824493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:03.826698 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2132b742-3830-4833-9be1-91c4bff21537 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:03.846721 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a418c-ca84-43b6-a227-12486b8cbe2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:03.847641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a418c-ca84-43b6-a227-12486b8cbe2f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:03.848141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a418c-ca84-43b6-a227-12486b8cbe2f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:03.848630 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a418c-ca84-43b6-a227-12486b8cbe2f None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:03.849500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a418c-ca84-43b6-a227-12486b8cbe2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:03.851720 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a418c-ca84-43b6-a227-12486b8cbe2f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:03.865684 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 475/930] 10.222.0.39 () {56 vars in 1087 bytes} [Thu Jul 28 22:16:03 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:16:03.869984 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fd221-00cf-49c2-a221-81dd54a85f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:03.870627 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fd221-00cf-49c2-a221-81dd54a85f49 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:03.870907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fd221-00cf-49c2-a221-81dd54a85f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:03.871177 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fd221-00cf-49c2-a221-81dd54a85f49 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:03.871914 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fd221-00cf-49c2-a221-81dd54a85f49 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:03.873454 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37fd221-00cf-49c2-a221-81dd54a85f49 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:03.913584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 456/931] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:16:03 2022] POST /identity/v3/endpoints => generated 347 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:16:04.068977 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b065ef5-8f27-42c2-b742-f93b19dfd470 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:04.069270 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b065ef5-8f27-42c2-b742-f93b19dfd470 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:04.069470 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b065ef5-8f27-42c2-b742-f93b19dfd470 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:04.071823 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 476/932] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:04.077369 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d61c63-f9a4-4117-80e2-0239f1143a6e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:04.077693 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d61c63-f9a4-4117-80e2-0239f1143a6e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:04.077968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d61c63-f9a4-4117-80e2-0239f1143a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:04.098339 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-92d61c63-f9a4-4117-80e2-0239f1143a6e None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:04.099603 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-92d61c63-f9a4-4117-80e2-0239f1143a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:04.179996 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 457/933] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:16:04.202895 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b065ef5-8f27-42c2-b742-f93b19dfd470 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:04.318309 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b065ef5-8f27-42c2-b742-f93b19dfd470 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:04.357683 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1953206-d468-4693-be35-5235db7299fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:04.358662 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1953206-d468-4693-be35-5235db7299fb None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:04.359149 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1953206-d468-4693-be35-5235db7299fb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:04.359624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1953206-d468-4693-be35-5235db7299fb None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:04.360526 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1953206-d468-4693-be35-5235db7299fb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:04.362183 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1953206-d468-4693-be35-5235db7299fb None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:04.365022 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: WARNING keystone.server.flask.common [None req-d1953206-d468-4693-be35-5235db7299fb None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 28 22:16:04.384385 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.sql.core [None req-d1953206-d468-4693-be35-5235db7299fb None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 22:16:04.384385 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 22:16:04.384385 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [parameters: {'id': 'e847b30419234ef98ef00268e9c4164d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 22:16:04.384385 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80362) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 22:16:04.393583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-d1953206-d468-4693-be35-5235db7299fb None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e847b30419234ef98ef00268e9c4164d.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e847b30419234ef98ef00268e9c4164d. Jul 28 22:16:04.393583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:04.393583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 22:16:04.393583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 22:16:04.393583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 22:16:04.393583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 22:16:04.393583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 22:16:04.393583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 22:16:04.393583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 22:16:04.393583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application conn.query(q) Jul 28 22:16:04.393583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 22:16:04.393583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 22:16:04.393583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 22:16:04.394397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application result.read() Jul 28 22:16:04.394397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 22:16:04.394397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 22:16:04.394397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 22:16:04.394397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 22:16:04.394397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 22:16:04.394397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 22:16:04.394397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 22:16:04.394397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 22:16:04.394397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 22:16:04.394397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:04.394397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 22:16:04.394397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:04.394397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:04.394397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 22:16:04.394397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 22:16:04.394397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 28 22:16:04.394397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 28 22:16:04.395443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 22:16:04.395443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application next(self.gen) Jul 28 22:16:04.395443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 22:16:04.395443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application yield resource Jul 28 22:16:04.395443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 22:16:04.395443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application next(self.gen) Jul 28 22:16:04.395443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 22:16:04.395443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 22:16:04.395443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 22:16:04.395443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 22:16:04.395443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 22:16:04.395443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise self.value Jul 28 22:16:04.395443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 22:16:04.395443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 22:16:04.395443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 22:16:04.395443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application session.commit() Jul 28 22:16:04.395443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 22:16:04.396380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 22:16:04.396380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 22:16:04.396380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 22:16:04.396380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 22:16:04.396380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self.session.flush() Jul 28 22:16:04.396380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 22:16:04.396380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 22:16:04.396380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 22:16:04.396380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 22:16:04.396380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 22:16:04.396380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application compat.raise_( Jul 28 22:16:04.396380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 22:16:04.396380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception Jul 28 22:16:04.396380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 22:16:04.396380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 22:16:04.396380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 22:16:04.396380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 22:16:04.397291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 22:16:04.397291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 22:16:04.397291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 22:16:04.397291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 22:16:04.397291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 28 22:16:04.397291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 28 22:16:04.397291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 22:16:04.397291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 22:16:04.397291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 22:16:04.397291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 22:16:04.397291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 22:16:04.397291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 22:16:04.397291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 22:16:04.397291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 22:16:04.397291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 22:16:04.400059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 22:16:04.400059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 22:16:04.400059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception Jul 28 22:16:04.400059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 22:16:04.400059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 22:16:04.400059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 22:16:04.400059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 22:16:04.400059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 22:16:04.400059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 22:16:04.400059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 22:16:04.400059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application conn.query(q) Jul 28 22:16:04.400059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 22:16:04.400059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 22:16:04.400059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 22:16:04.400059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application result.read() Jul 28 22:16:04.400059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 22:16:04.400059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 22:16:04.401026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 22:16:04.401026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 22:16:04.401026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 22:16:04.401026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 22:16:04.401026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 22:16:04.401026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 22:16:04.401026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 28 22:16:04.401026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 28 22:16:04.401026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [parameters: {'id': 'e847b30419234ef98ef00268e9c4164d', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 28 22:16:04.401026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 22:16:04.401026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:04.401026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:16:04.401026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:04.401026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:04.401026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 28 22:16:04.402542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 28 22:16:04.402542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 22:16:04.402542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 22:16:04.402542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 28 22:16:04.402542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:04.402542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:16:04.402542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:04.402542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:04.402542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:04.402542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:04.402542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:04.402542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:04.402542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:04.402542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:04.402542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:04.402542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:04.402542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:04.403494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:04.403494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 28 22:16:04.403494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 28 22:16:04.403494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:04.403494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:04.403494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 28 22:16:04.403494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 28 22:16:04.403494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e847b30419234ef98ef00268e9c4164d. Jul 28 22:16:04.403494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:04.406256 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 477/934] 10.222.0.39 () {60 vars in 1140 bytes} [Thu Jul 28 22:16:04 2022] POST /identity/v3/projects => generated 262 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:04.407603 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d61c63-f9a4-4117-80e2-0239f1143a6e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:04.410697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:04.519026 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d61c63-f9a4-4117-80e2-0239f1143a6e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:04.555557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ff0925-0e42-4317-8b04-a5b9273cd9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:04.556253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff0925-0e42-4317-8b04-a5b9273cd9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:04.556531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff0925-0e42-4317-8b04-a5b9273cd9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:04.556838 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ff0925-0e42-4317-8b04-a5b9273cd9c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:04.563766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff0925-0e42-4317-8b04-a5b9273cd9c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:04.565269 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ff0925-0e42-4317-8b04-a5b9273cd9c9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:04.579336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c2ff0925-0e42-4317-8b04-a5b9273cd9c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:16:04.579336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:04.579336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:04.579336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:04.579336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:04.579336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:04.579336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:04.579336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:04.579336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:04.579336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:04.579336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:04.579336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:04.579336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:16:04.579336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:16:04.579336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:16:04.580261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:16:04.580261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:04.580261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:04.580261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:04.580261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:04.580261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:04.580261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:04.580261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:04.580261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:04.580261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:04.580261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:04.580261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:04.580261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:04.580261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:04.580261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:04.580261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:04.580261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:04.581236 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:16:04.581236 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:16:04.581236 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:16:04.581236 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:16:04.581236 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:16:04.581236 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:16:04.581236 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:16:04.581236 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:04.585216 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 458/935] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:16:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:04.586953 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1953206-d468-4693-be35-5235db7299fb None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:04.637954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:04.702044 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1953206-d468-4693-be35-5235db7299fb None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:04.737337 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f1d71-9183-4e28-8db0-1eea6abb06de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:04.738040 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f1d71-9183-4e28-8db0-1eea6abb06de None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:04.738322 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f1d71-9183-4e28-8db0-1eea6abb06de None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:04.738591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f1d71-9183-4e28-8db0-1eea6abb06de None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:04.739416 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f1d71-9183-4e28-8db0-1eea6abb06de None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:04.739725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f1d71-9183-4e28-8db0-1eea6abb06de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:04.741360 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2f1d71-9183-4e28-8db0-1eea6abb06de None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:04.769747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 478/936] 10.222.0.39 () {56 vars in 1112 bytes} [Thu Jul 28 22:16:04 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:05.258333 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d873f53-e5d0-4422-a927-c221c56b359e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:05.258846 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d873f53-e5d0-4422-a927-c221c56b359e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:05.258846 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d873f53-e5d0-4422-a927-c221c56b359e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:05.260480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 459/937] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:05.266031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d70ccc-f5ca-4ae7-b89c-08d428d7393d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:05.266445 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d70ccc-f5ca-4ae7-b89c-08d428d7393d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:05.266618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d70ccc-f5ca-4ae7-b89c-08d428d7393d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:05.285787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e9d70ccc-f5ca-4ae7-b89c-08d428d7393d None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:05.287305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e9d70ccc-f5ca-4ae7-b89c-08d428d7393d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:05.362698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 479/938] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:05.384617 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d873f53-e5d0-4422-a927-c221c56b359e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:05.452788 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:05.495550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d873f53-e5d0-4422-a927-c221c56b359e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:05.532936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e39b51-730a-4082-9d37-d3f31f50835c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:05.533676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e39b51-730a-4082-9d37-d3f31f50835c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:05.533958 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e39b51-730a-4082-9d37-d3f31f50835c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:05.534233 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e39b51-730a-4082-9d37-d3f31f50835c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:05.546405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e39b51-730a-4082-9d37-d3f31f50835c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:16:05.551422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a4e39b51-730a-4082-9d37-d3f31f50835c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 22:16:05.551422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:05.551422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:05.551422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:05.551422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:05.551422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:05.551422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:05.551422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:05.551422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:05.551422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:05.551422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:05.551422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:05.551422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:16:05.551422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:16:05.551422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:16:05.551422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:16:05.552373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:16:05.552373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:16:05.552373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:16:05.552373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:16:05.552373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:05.552373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:05.552373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:05.552373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:05.552373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:05.552373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:05.552373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:05.552373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:05.552373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:05.552373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:05.552373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:05.552373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:05.553239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:05.553239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:05.553239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:05.553239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:05.553239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:16:05.553239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:16:05.553239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:16:05.553239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:16:05.553239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:16:05.553239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:16:05.553239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 22:16:05.553239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:05.553239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 460/939] 10.222.0.39 () {56 vars in 1120 bytes} [Thu Jul 28 22:16:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:05.558588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d70ccc-f5ca-4ae7-b89c-08d428d7393d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:05.588282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:05.673472 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d70ccc-f5ca-4ae7-b89c-08d428d7393d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:05.709891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102e830-05f6-4834-941a-656a208dbffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:05.711083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102e830-05f6-4834-941a-656a208dbffe None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:05.711384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102e830-05f6-4834-941a-656a208dbffe None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:05.711715 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102e830-05f6-4834-941a-656a208dbffe None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:05.712454 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102e830-05f6-4834-941a-656a208dbffe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:05.712764 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102e830-05f6-4834-941a-656a208dbffe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:05.714186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102e830-05f6-4834-941a-656a208dbffe None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:05.723445 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 480/940] 10.222.0.39 () {56 vars in 1115 bytes} [Thu Jul 28 22:16:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:05.730601 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e39b51-730a-4082-9d37-d3f31f50835c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:05.836473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e39b51-730a-4082-9d37-d3f31f50835c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:05.862552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:05.873929 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8f1c1b-11c7-42ea-91a4-ada3dc4b2c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:05.875104 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f1c1b-11c7-42ea-91a4-ada3dc4b2c5b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:05.875393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f1c1b-11c7-42ea-91a4-ada3dc4b2c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:05.875669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8f1c1b-11c7-42ea-91a4-ada3dc4b2c5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:05.887131 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f1c1b-11c7-42ea-91a4-ada3dc4b2c5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:05.888848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8f1c1b-11c7-42ea-91a4-ada3dc4b2c5b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:05.904198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-2e8f1c1b-11c7-42ea-91a4-ada3dc4b2c5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:05.904198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:05.904198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:05.904198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:05.904198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:05.904198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:05.904198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:05.904198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:05.904198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:05.904198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:05.904198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:05.904198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:05.904198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:16:05.904198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:16:05.904198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:16:05.905151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:16:05.905151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:05.905151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:05.905151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:05.905151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:05.905151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:05.905151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:05.905151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:05.905151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:05.905151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:05.905151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:05.905151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:05.905151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:05.905151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:05.905151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:05.905151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:05.905151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:05.906094 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:16:05.906094 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:16:05.906094 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:05.906094 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:05.908910 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 461/941] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:05.911383 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102e830-05f6-4834-941a-656a208dbffe None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:05.978185 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d5a82-6325-4d09-9258-48eea4eded0c None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:05.978487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d5a82-6325-4d09-9258-48eea4eded0c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:05.978678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d5a82-6325-4d09-9258-48eea4eded0c None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:05.980256 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 462/942] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:16:05.985460 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43288a7-1b6b-41ab-b19b-eea3a3244969 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:05.985777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43288a7-1b6b-41ab-b19b-eea3a3244969 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:05.986052 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43288a7-1b6b-41ab-b19b-eea3a3244969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:06.007915 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b43288a7-1b6b-41ab-b19b-eea3a3244969 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:06.009162 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b43288a7-1b6b-41ab-b19b-eea3a3244969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:06.015220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:06.031950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102e830-05f6-4834-941a-656a208dbffe None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:06.078349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421f479-6ae6-42c1-a9a8-84db48b42d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:06.079952 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421f479-6ae6-42c1-a9a8-84db48b42d43 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:06.080673 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421f479-6ae6-42c1-a9a8-84db48b42d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:06.081251 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421f479-6ae6-42c1-a9a8-84db48b42d43 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:06.082424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421f479-6ae6-42c1-a9a8-84db48b42d43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:06.083351 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421f479-6ae6-42c1-a9a8-84db48b42d43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:06.086040 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421f479-6ae6-42c1-a9a8-84db48b42d43 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:06.097667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 463/943] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:16:06.108160 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 481/944] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:06.113595 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43288a7-1b6b-41ab-b19b-eea3a3244969 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:06.120920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421f479-6ae6-42c1-a9a8-84db48b42d43 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:06.228999 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43288a7-1b6b-41ab-b19b-eea3a3244969 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:06.229511 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421f479-6ae6-42c1-a9a8-84db48b42d43 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:06.237993 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:06.266532 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716d7fc9-13ae-44f3-bbab-944b36584f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:06.267173 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d7fc9-13ae-44f3-bbab-944b36584f3a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:06.267451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d7fc9-13ae-44f3-bbab-944b36584f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:06.267728 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716d7fc9-13ae-44f3-bbab-944b36584f3a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:06.268008 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1dc7fc-8c5e-444a-a230-fe6b472d8bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:06.268396 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d7fc9-13ae-44f3-bbab-944b36584f3a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:06.268649 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1dc7fc-8c5e-444a-a230-fe6b472d8bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:06.268930 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1dc7fc-8c5e-444a-a230-fe6b472d8bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:06.269204 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1dc7fc-8c5e-444a-a230-fe6b472d8bf5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:06.269789 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716d7fc9-13ae-44f3-bbab-944b36584f3a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:06.278115 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1dc7fc-8c5e-444a-a230-fe6b472d8bf5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:06.279555 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1dc7fc-8c5e-444a-a230-fe6b472d8bf5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:06.279685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-716d7fc9-13ae-44f3-bbab-944b36584f3a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 22:16:06.279685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:06.279685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:16:06.279685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:16:06.279685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:16:06.279685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:16:06.279685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:16:06.279685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:06.279685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 22:16:06.279685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:06.279685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:16:06.279685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:06.279685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:06.279685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:06.279685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:06.279685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:06.279685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:06.280637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:06.280637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:06.280637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:06.280637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:06.280637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:06.280637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:06.280637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:16:06.280637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:16:06.280637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:16:06.280637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:16:06.280637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:06.280637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:06.280637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:06.280637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:06.280637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:06.280637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:06.281505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:06.281505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:06.281505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:06.281505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:06.281505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:06.281505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:06.281505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:06.281505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:06.281505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:06.281505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:06.281505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:16:06.281505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:06.281505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 22:16:06.281505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:06.285844 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 482/945] 10.222.0.39 () {56 vars in 1126 bytes} [Thu Jul 28 22:16:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:06.287072 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac31d0-0cfb-4038-94c0-b55d660f6c99 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:06.287358 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac31d0-0cfb-4038-94c0-b55d660f6c99 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:06.287637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eac31d0-0cfb-4038-94c0-b55d660f6c99 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:06.293592 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 483/946] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:06.293592 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac31d0-0cfb-4038-94c0-b55d660f6c99 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:06.295299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-2c1dc7fc-8c5e-444a-a230-fe6b472d8bf5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 22:16:06.295299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:06.295299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:06.295299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:06.295299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:06.295299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:06.295299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:06.295299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:06.295299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:06.295299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:06.295299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:06.295299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:06.295299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:16:06.295299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:16:06.295299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:16:06.295299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:16:06.297599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:06.297599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:06.297599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:16:06.297599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:16:06.297599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:16:06.297599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:16:06.297599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:06.297599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:06.297599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:06.297599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:06.297599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:06.297599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:06.297599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:06.297599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:06.297599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:06.297599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:06.297599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:06.297599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:06.299848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:06.299848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:06.299848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:16:06.299848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:16:06.299848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:16:06.299848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:16:06.299848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:16:06.299848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:16:06.299848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 22:16:06.299848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:06.302035 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 464/947] 10.222.0.39 () {56 vars in 1117 bytes} [Thu Jul 28 22:16:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:06.302586 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95204376-6b37-4249-a3fa-9fed5155c0dd None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:06.302839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95204376-6b37-4249-a3fa-9fed5155c0dd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:06.303118 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95204376-6b37-4249-a3fa-9fed5155c0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:06.315704 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:06.322319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-95204376-6b37-4249-a3fa-9fed5155c0dd None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:06.323526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-95204376-6b37-4249-a3fa-9fed5155c0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:06.397769 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 465/948] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:06.397769 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95204376-6b37-4249-a3fa-9fed5155c0dd None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:06.420799 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eac31d0-0cfb-4038-94c0-b55d660f6c99 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:06.458770 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b747d-ccc6-4578-9e93-5cba7d4be627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:06.459385 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b747d-ccc6-4578-9e93-5cba7d4be627 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:06.459673 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b747d-ccc6-4578-9e93-5cba7d4be627 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:06.460109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b747d-ccc6-4578-9e93-5cba7d4be627 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:06.460732 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b747d-ccc6-4578-9e93-5cba7d4be627 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:06.461047 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b747d-ccc6-4578-9e93-5cba7d4be627 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:06.462449 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b747d-ccc6-4578-9e93-5cba7d4be627 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:06.467280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 484/949] 10.222.0.39 () {56 vars in 1122 bytes} [Thu Jul 28 22:16:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:06.472634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b747d-ccc6-4578-9e93-5cba7d4be627 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:06.498125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:06.504271 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95204376-6b37-4249-a3fa-9fed5155c0dd None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:06.541628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f019c3-e4b0-4253-aea1-dfe47dd8ba78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:06.542333 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f019c3-e4b0-4253-aea1-dfe47dd8ba78 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:06.542677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f019c3-e4b0-4253-aea1-dfe47dd8ba78 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:06.543012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f019c3-e4b0-4253-aea1-dfe47dd8ba78 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:06.543873 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f019c3-e4b0-4253-aea1-dfe47dd8ba78 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:06.544258 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f019c3-e4b0-4253-aea1-dfe47dd8ba78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:06.545774 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f019c3-e4b0-4253-aea1-dfe47dd8ba78 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:06.560875 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:06.565862 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 466/950] 10.222.0.39 () {56 vars in 1149 bytes} [Thu Jul 28 22:16:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:06.567114 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f019c3-e4b0-4253-aea1-dfe47dd8ba78 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:06.590947 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b747d-ccc6-4578-9e93-5cba7d4be627 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:06.628735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf198998-5aae-4ce6-b5c5-7ecab419bde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:06.629378 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf198998-5aae-4ce6-b5c5-7ecab419bde4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:06.629694 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf198998-5aae-4ce6-b5c5-7ecab419bde4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:06.629974 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf198998-5aae-4ce6-b5c5-7ecab419bde4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:06.630604 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf198998-5aae-4ce6-b5c5-7ecab419bde4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:06.631990 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf198998-5aae-4ce6-b5c5-7ecab419bde4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:06.641367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-cf198998-5aae-4ce6-b5c5-7ecab419bde4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 22:16:06.641367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:06.641367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:16:06.641367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:16:06.641367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:16:06.641367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:16:06.641367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:16:06.641367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:06.641367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 22:16:06.641367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:06.641367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:16:06.641367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:06.641367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:06.641367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:06.641367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:06.641367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:06.641367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:06.647535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:06.647535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:06.647535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:06.647535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:06.647535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:06.647535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:06.647535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:16:06.647535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:16:06.647535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:16:06.647535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:16:06.647535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:06.647535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:06.647535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:06.647535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:06.647535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:06.647535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:06.649327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:06.649327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:06.649327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:06.649327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:06.649327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:06.649327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:06.649327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:06.649327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:06.649327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:06.649327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:06.649327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:16:06.649327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:06.649327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 22:16:06.649327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:06.649327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 485/951] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:16:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:06.650347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf198998-5aae-4ce6-b5c5-7ecab419bde4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:06.677276 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f019c3-e4b0-4253-aea1-dfe47dd8ba78 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:06.720372 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a170de2-a835-444c-ab98-21751f2b6527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:06.721095 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a170de2-a835-444c-ab98-21751f2b6527 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:06.721374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a170de2-a835-444c-ab98-21751f2b6527 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:06.721672 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a170de2-a835-444c-ab98-21751f2b6527 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:06.722346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a170de2-a835-444c-ab98-21751f2b6527 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:06.722644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a170de2-a835-444c-ab98-21751f2b6527 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:06.723993 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a170de2-a835-444c-ab98-21751f2b6527 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:06.733748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 467/952] 10.222.0.39 () {56 vars in 1122 bytes} [Thu Jul 28 22:16:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:06.733748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a170de2-a835-444c-ab98-21751f2b6527 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:06.754058 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:06.762994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf198998-5aae-4ce6-b5c5-7ecab419bde4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:06.800832 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b1d59-fd3a-41c7-9d6e-050ca0ef67ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:06.801498 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b1d59-fd3a-41c7-9d6e-050ca0ef67ab None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:06.801806 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b1d59-fd3a-41c7-9d6e-050ca0ef67ab None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:06.802087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b1d59-fd3a-41c7-9d6e-050ca0ef67ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:06.812649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:06.815856 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b1d59-fd3a-41c7-9d6e-050ca0ef67ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:06.817816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49b1d59-fd3a-41c7-9d6e-050ca0ef67ab None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:06.834052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-f49b1d59-fd3a-41c7-9d6e-050ca0ef67ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:06.834052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:06.834052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:06.834052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:06.834052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:06.834052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:06.834052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:06.834052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:06.834052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:06.834052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:06.834052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:06.834052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:06.834052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:16:06.834052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:16:06.834052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:16:06.834996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:16:06.834996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:06.834996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:06.834996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:06.834996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:06.834996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:06.834996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:06.834996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:06.834996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:06.834996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:06.834996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:06.834996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:06.834996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:06.834996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:06.834996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:06.834996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:06.834996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:06.836655 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:16:06.836655 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:16:06.836655 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:06.836655 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:06.836655 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 486/953] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:06.836655 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b1d59-fd3a-41c7-9d6e-050ca0ef67ab None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:06.848710 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a170de2-a835-444c-ab98-21751f2b6527 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:06.892314 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942b8e8-efcf-4d06-97cf-6a218b6368da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:06.892935 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942b8e8-efcf-4d06-97cf-6a218b6368da None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:06.893229 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942b8e8-efcf-4d06-97cf-6a218b6368da None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:06.893483 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942b8e8-efcf-4d06-97cf-6a218b6368da None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:06.894197 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942b8e8-efcf-4d06-97cf-6a218b6368da None admin] RBAC: Adding query filter params (name=compute) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:06.894503 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942b8e8-efcf-4d06-97cf-6a218b6368da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:06.895867 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942b8e8-efcf-4d06-97cf-6a218b6368da None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:06.905378 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 468/954] 10.222.0.39 () {56 vars in 1112 bytes} [Thu Jul 28 22:16:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:06.905378 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942b8e8-efcf-4d06-97cf-6a218b6368da None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:06.944508 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b1d59-fd3a-41c7-9d6e-050ca0ef67ab None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:06.979494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c4572-cffd-4256-a067-b88c2dbc8e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:06.980120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c4572-cffd-4256-a067-b88c2dbc8e6a None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:06.980408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c4572-cffd-4256-a067-b88c2dbc8e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:06.980688 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c4572-cffd-4256-a067-b88c2dbc8e6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:06.981405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c4572-cffd-4256-a067-b88c2dbc8e6a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:06.982825 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c4572-cffd-4256-a067-b88c2dbc8e6a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:06.989905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:07.004658 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 487/955] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:16:06 2022] POST /identity/v3/endpoints => generated 349 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:16:07.005404 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c4572-cffd-4256-a067-b88c2dbc8e6a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:07.010819 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942b8e8-efcf-4d06-97cf-6a218b6368da None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:07.059029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735647c7-3d10-4b1f-8fa1-afe7a02e5fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:07.059940 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735647c7-3d10-4b1f-8fa1-afe7a02e5fed None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:07.060436 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735647c7-3d10-4b1f-8fa1-afe7a02e5fed None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:07.060701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735647c7-3d10-4b1f-8fa1-afe7a02e5fed None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:07.061490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735647c7-3d10-4b1f-8fa1-afe7a02e5fed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:07.061844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735647c7-3d10-4b1f-8fa1-afe7a02e5fed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:07.063273 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735647c7-3d10-4b1f-8fa1-afe7a02e5fed None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:07.063424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:07.077720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 469/956] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:07.079285 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee79ba-657c-4434-aa8f-940ad24ddaa6 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:07.079631 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee79ba-657c-4434-aa8f-940ad24ddaa6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:07.079839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fee79ba-657c-4434-aa8f-940ad24ddaa6 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:07.081071 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 470/957] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:07.082995 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee79ba-657c-4434-aa8f-940ad24ddaa6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:07.130459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c4572-cffd-4256-a067-b88c2dbc8e6a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:07.178682 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a410b-7ebc-40d7-9b78-a968b1eb5cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:07.180308 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a410b-7ebc-40d7-9b78-a968b1eb5cae None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:07.180456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a410b-7ebc-40d7-9b78-a968b1eb5cae None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:07.181036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a410b-7ebc-40d7-9b78-a968b1eb5cae None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:07.181774 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a410b-7ebc-40d7-9b78-a968b1eb5cae None admin] RBAC: Adding query filter params (type=compute) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:07.182422 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a410b-7ebc-40d7-9b78-a968b1eb5cae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:07.183805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a410b-7ebc-40d7-9b78-a968b1eb5cae None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:07.188803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 488/958] 10.222.0.39 () {56 vars in 1112 bytes} [Thu Jul 28 22:16:07 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:07.190601 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a2784-02cf-40c5-98f8-5e73e68601c1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:07.190894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a2784-02cf-40c5-98f8-5e73e68601c1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:07.191174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47a2784-02cf-40c5-98f8-5e73e68601c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:07.212050 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fee79ba-657c-4434-aa8f-940ad24ddaa6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:07.212409 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c47a2784-02cf-40c5-98f8-5e73e68601c1 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:07.213703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c47a2784-02cf-40c5-98f8-5e73e68601c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:07.256146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598c9bc-7c2f-4bc5-a21a-4328bb913553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:07.256733 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598c9bc-7c2f-4bc5-a21a-4328bb913553 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:07.257017 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598c9bc-7c2f-4bc5-a21a-4328bb913553 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:07.257628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598c9bc-7c2f-4bc5-a21a-4328bb913553 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:07.266501 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598c9bc-7c2f-4bc5-a21a-4328bb913553 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:07.268078 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598c9bc-7c2f-4bc5-a21a-4328bb913553 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:07.270749 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:07.283908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a598c9bc-7c2f-4bc5-a21a-4328bb913553 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:16:07.283908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:07.283908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:07.283908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:07.283908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:07.283908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:07.283908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:07.283908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:07.283908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:07.283908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:07.283908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:07.283908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:07.283908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:16:07.283908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:16:07.283908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:16:07.284968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:16:07.284968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:07.284968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:07.284968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:07.284968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:07.284968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:07.284968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:07.284968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:07.284968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:07.284968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:07.284968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:07.284968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:07.284968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:07.284968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:07.284968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:07.284968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:07.284968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:07.286468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:16:07.286468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:16:07.286468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:16:07.286468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:16:07.286468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:16:07.286468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:16:07.286468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:16:07.286468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:07.286468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 471/959] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:16:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:07.292528 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598c9bc-7c2f-4bc5-a21a-4328bb913553 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:07.306438 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 489/960] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:07.332153 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a2784-02cf-40c5-98f8-5e73e68601c1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:07.403268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:07.417074 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598c9bc-7c2f-4bc5-a21a-4328bb913553 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:07.452459 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa41374-c989-4dca-90c2-bbabc534d29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:07.453369 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa41374-c989-4dca-90c2-bbabc534d29c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:07.453906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa41374-c989-4dca-90c2-bbabc534d29c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:07.454347 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa41374-c989-4dca-90c2-bbabc534d29c None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:07.455309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa41374-c989-4dca-90c2-bbabc534d29c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:07.455766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa41374-c989-4dca-90c2-bbabc534d29c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:07.457363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa41374-c989-4dca-90c2-bbabc534d29c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:07.470522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 472/961] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:16:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:07.473419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47a2784-02cf-40c5-98f8-5e73e68601c1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:07.477094 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa41374-c989-4dca-90c2-bbabc534d29c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:07.520208 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8b6cf-d53b-4ed1-9c5b-dd5dc13e920d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:07.521203 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8b6cf-d53b-4ed1-9c5b-dd5dc13e920d None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:07.521764 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8b6cf-d53b-4ed1-9c5b-dd5dc13e920d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:07.522238 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f8b6cf-d53b-4ed1-9c5b-dd5dc13e920d None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:07.523886 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8b6cf-d53b-4ed1-9c5b-dd5dc13e920d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:07.525610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f8b6cf-d53b-4ed1-9c5b-dd5dc13e920d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:07.542221 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.sql.core [None req-a3f8b6cf-d53b-4ed1-9c5b-dd5dc13e920d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 22:16:07.542221 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 22:16:07.542221 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [parameters: {'user_id': 'b03a2134fe53402fac28dfe47c3160be', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 22:16:07.542221 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80362) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 28 22:16:07.547076 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:07.551617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-a3f8b6cf-d53b-4ed1-9c5b-dd5dc13e920d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 22:16:07.551617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:07.551617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 22:16:07.551617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 22:16:07.551617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 22:16:07.551617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 22:16:07.551617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 22:16:07.551617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 22:16:07.551617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 22:16:07.551617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application conn.query(q) Jul 28 22:16:07.551617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 22:16:07.551617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 22:16:07.551617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 22:16:07.551617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application result.read() Jul 28 22:16:07.551617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 22:16:07.552724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 22:16:07.552724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 22:16:07.552724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 22:16:07.552724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 22:16:07.552724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 22:16:07.552724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 22:16:07.552724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 22:16:07.552724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 22:16:07.552724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:07.552724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 28 22:16:07.552724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:07.552724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:07.552724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 28 22:16:07.552724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 28 22:16:07.552724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 28 22:16:07.552724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 28 22:16:07.552724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 22:16:07.552724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application next(self.gen) Jul 28 22:16:07.554599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 28 22:16:07.554599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application yield resource Jul 28 22:16:07.554599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 28 22:16:07.554599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application next(self.gen) Jul 28 22:16:07.554599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 28 22:16:07.554599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self.session.rollback() Jul 28 22:16:07.554599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 28 22:16:07.554599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self.force_reraise() Jul 28 22:16:07.554599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 28 22:16:07.554599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise self.value Jul 28 22:16:07.554599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 28 22:16:07.554599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 28 22:16:07.554599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 28 22:16:07.554599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application session.commit() Jul 28 22:16:07.554599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 28 22:16:07.554599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 28 22:16:07.554599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 28 22:16:07.555811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._prepare_impl() Jul 28 22:16:07.555811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 28 22:16:07.555811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self.session.flush() Jul 28 22:16:07.555811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 28 22:16:07.555811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._flush(objects) Jul 28 22:16:07.555811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 28 22:16:07.555811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 28 22:16:07.555811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 28 22:16:07.555811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application compat.raise_( Jul 28 22:16:07.555811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 22:16:07.555811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception Jul 28 22:16:07.555811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 28 22:16:07.555811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application flush_context.execute() Jul 28 22:16:07.555811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 28 22:16:07.555811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rec.execute(self) Jul 28 22:16:07.555811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 28 22:16:07.555811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 28 22:16:07.556955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 28 22:16:07.556955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 28 22:16:07.556955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 28 22:16:07.556955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 28 22:16:07.556955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 28 22:16:07.556955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 28 22:16:07.556955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 28 22:16:07.556955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 28 22:16:07.556955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 28 22:16:07.556955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 28 22:16:07.556955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 28 22:16:07.556955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 28 22:16:07.556955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 28 22:16:07.556955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 28 22:16:07.556955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 28 22:16:07.556955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception Jul 28 22:16:07.558224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 28 22:16:07.558224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 28 22:16:07.558224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 28 22:16:07.558224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 28 22:16:07.558224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 28 22:16:07.558224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application result = self._query(query) Jul 28 22:16:07.558224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 28 22:16:07.558224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application conn.query(q) Jul 28 22:16:07.558224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 28 22:16:07.558224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 28 22:16:07.558224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 28 22:16:07.558224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application result.read() Jul 28 22:16:07.558224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 28 22:16:07.558224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 28 22:16:07.558224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 28 22:16:07.558224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 28 22:16:07.559306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 28 22:16:07.559306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 28 22:16:07.559306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 28 22:16:07.559306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 28 22:16:07.559306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 28 22:16:07.559306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 28 22:16:07.559306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b03a2134fe53402fac28dfe47c3160be', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 28 22:16:07.559306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 28 22:16:07.559306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:07.559306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:16:07.559306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:07.559306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:07.559306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:07.559306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:07.559306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:07.560349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:07.560349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:07.560349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:07.560349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:07.560349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:07.560349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:07.560349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:07.560349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 28 22:16:07.560349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 28 22:16:07.560349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:07.560349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:07.560349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:16:07.560349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:16:07.560349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:16:07.560349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:16:07.560349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 28 22:16:07.560349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 28 22:16:07.561482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 28 22:16:07.561482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 28 22:16:07.561482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 28 22:16:07.561482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 28 22:16:07.561482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 28 22:16:07.561482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:07.568639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 490/962] 10.222.0.39 () {60 vars in 1131 bytes} [Thu Jul 28 22:16:07 2022] POST /identity/v3/users => generated 160 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:07.570610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8b6cf-d53b-4ed1-9c5b-dd5dc13e920d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:07.613150 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa41374-c989-4dca-90c2-bbabc534d29c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:07.658507 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a6523-ba97-47a2-ba84-f33b6596831a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:07.660025 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a6523-ba97-47a2-ba84-f33b6596831a None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:07.660519 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a6523-ba97-47a2-ba84-f33b6596831a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:07.661394 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a6523-ba97-47a2-ba84-f33b6596831a None admin] PATH_INFO: `/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/85626350279c40ea99da93b9ed6994b2/roles/62bb6d1764134942a1069bf3a5c1e0c4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:07.664571 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a6523-ba97-47a2-ba84-f33b6596831a None admin] RBAC: Authorizing `identity:create_grant(project_id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user_id=85626350279c40ea99da93b9ed6994b2, role_id=62bb6d1764134942a1069bf3a5c1e0c4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:07.666978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a6523-ba97-47a2-ba84-f33b6596831a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:07.686564 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 473/963] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:16:07 2022] PUT /identity/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/85626350279c40ea99da93b9ed6994b2/roles/62bb6d1764134942a1069bf3a5c1e0c4 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:16:07.687974 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f8b6cf-d53b-4ed1-9c5b-dd5dc13e920d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:07.691591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:07.736967 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e1270-b0d5-4e0a-a9bb-d0b08f13c175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:07.737853 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e1270-b0d5-4e0a-a9bb-d0b08f13c175 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:07.738138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e1270-b0d5-4e0a-a9bb-d0b08f13c175 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:07.738416 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e1270-b0d5-4e0a-a9bb-d0b08f13c175 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:07.747905 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e1270-b0d5-4e0a-a9bb-d0b08f13c175 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:07.749622 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e1270-b0d5-4e0a-a9bb-d0b08f13c175 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:07.764009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-7b5e1270-b0d5-4e0a-a9bb-d0b08f13c175 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 22:16:07.764009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:07.764009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:07.764009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:07.764009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:07.764009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:07.764009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:07.764009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:07.764009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:07.764009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:07.764009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:07.764009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:07.764009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:16:07.764009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:16:07.764009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:16:07.764009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:16:07.765053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:07.765053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:07.765053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:16:07.765053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:16:07.765053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:16:07.765053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:16:07.765053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:07.765053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:07.765053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:07.765053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:07.765053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:07.765053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:07.765053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:07.765053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:07.765053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:07.765053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:07.765053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:07.765053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:07.766037 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:07.766037 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:07.766037 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:16:07.766037 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:16:07.766037 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:16:07.766037 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:16:07.766037 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:16:07.766037 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:16:07.766037 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 28 22:16:07.766037 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:07.769698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 491/964] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:16:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:07.771966 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a6523-ba97-47a2-ba84-f33b6596831a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:07.886642 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a6523-ba97-47a2-ba84-f33b6596831a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:07.906847 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:07.926254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e915b-6864-4acf-906f-bce1c1b7c2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:07.926974 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e915b-6864-4acf-906f-bce1c1b7c2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:07.927322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e915b-6864-4acf-906f-bce1c1b7c2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:07.927683 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e915b-6864-4acf-906f-bce1c1b7c2b5 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:07.928514 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e915b-6864-4acf-906f-bce1c1b7c2b5 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:07.928914 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e915b-6864-4acf-906f-bce1c1b7c2b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:07.930514 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0e915b-6864-4acf-906f-bce1c1b7c2b5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:07.973682 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 474/965] 10.222.0.39 () {56 vars in 1101 bytes} [Thu Jul 28 22:16:07 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:09.165000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83258f-30be-4127-a3a4-c933486cf448 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:09.165428 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83258f-30be-4127-a3a4-c933486cf448 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:09.165659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f83258f-30be-4127-a3a4-c933486cf448 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:09.169877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 492/966] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:09.175079 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af02a65-f31f-4512-a5a0-21960e07996c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:09.175585 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af02a65-f31f-4512-a5a0-21960e07996c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:09.175664 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af02a65-f31f-4512-a5a0-21960e07996c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:09.193766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0af02a65-f31f-4512-a5a0-21960e07996c None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:09.195004 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0af02a65-f31f-4512-a5a0-21960e07996c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:09.263865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 475/967] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:09.286769 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83258f-30be-4127-a3a4-c933486cf448 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:09.341426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ca920-d529-4ab0-aa09-20c31ebe12af None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:09.341771 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ca920-d529-4ab0-aa09-20c31ebe12af None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:09.342025 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5ca920-d529-4ab0-aa09-20c31ebe12af None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:09.343494 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 476/968] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:16:09.348162 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d91f8-9946-4e9c-8349-ae5cda9de4cb None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:09.348417 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d91f8-9946-4e9c-8349-ae5cda9de4cb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:09.348690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51d91f8-9946-4e9c-8349-ae5cda9de4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:09.368870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b51d91f8-9946-4e9c-8349-ae5cda9de4cb None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:09.370102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b51d91f8-9946-4e9c-8349-ae5cda9de4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:09.396266 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f83258f-30be-4127-a3a4-c933486cf448 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:09.435998 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1998bb-2cf9-40ca-827e-f1c610ab55d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:09.436559 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1998bb-2cf9-40ca-827e-f1c610ab55d1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:09.436841 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1998bb-2cf9-40ca-827e-f1c610ab55d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:09.437123 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1998bb-2cf9-40ca-827e-f1c610ab55d1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:09.437772 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1998bb-2cf9-40ca-827e-f1c610ab55d1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:09.439130 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1998bb-2cf9-40ca-827e-f1c610ab55d1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:09.445575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 477/969] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:16:09.448793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ec1998bb-2cf9-40ca-827e-f1c610ab55d1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 22:16:09.448793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:09.448793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:16:09.448793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:16:09.448793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:16:09.448793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:16:09.448793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:16:09.448793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:09.448793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 22:16:09.448793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:09.448793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:16:09.448793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:09.448793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:09.448793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:09.448793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:09.448793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:09.448793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:09.449906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:09.449906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:09.449906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:09.449906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:09.449906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:09.449906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:09.449906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:16:09.449906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:16:09.449906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:16:09.449906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:16:09.449906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:09.449906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:09.449906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:09.449906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:09.449906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:09.449906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:09.450816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:09.450816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:09.450816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:09.450816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:09.450816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:09.450816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:09.450816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:09.450816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:09.450816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:09.450816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:09.450816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:16:09.450816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:09.450816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 22:16:09.450816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:09.453867 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 493/970] 10.222.0.39 () {56 vars in 1126 bytes} [Thu Jul 28 22:16:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:09.458641 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d91f8-9946-4e9c-8349-ae5cda9de4cb None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:09.464735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1998bb-2cf9-40ca-827e-f1c610ab55d1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:09.495187 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:09.496424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:09.567998 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51d91f8-9946-4e9c-8349-ae5cda9de4cb None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:09.577542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1998bb-2cf9-40ca-827e-f1c610ab55d1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:09.604816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fbbf4-0f54-4d77-abc0-b53fc6bacd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:09.605632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fbbf4-0f54-4d77-abc0-b53fc6bacd92 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:09.606085 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fbbf4-0f54-4d77-abc0-b53fc6bacd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:09.606381 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8fbbf4-0f54-4d77-abc0-b53fc6bacd92 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:09.607160 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fbbf4-0f54-4d77-abc0-b53fc6bacd92 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:09.607566 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fbbf4-0f54-4d77-abc0-b53fc6bacd92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:09.609102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8fbbf4-0f54-4d77-abc0-b53fc6bacd92 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:09.614245 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 478/971] 10.222.0.39 () {56 vars in 1122 bytes} [Thu Jul 28 22:16:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:09.620290 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fbbf4-0f54-4d77-abc0-b53fc6bacd92 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:09.621069 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da34e66-633e-4f0a-88b6-5058bd71ec86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:09.621800 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da34e66-633e-4f0a-88b6-5058bd71ec86 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:09.622087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da34e66-633e-4f0a-88b6-5058bd71ec86 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:09.622370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da34e66-633e-4f0a-88b6-5058bd71ec86 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:09.623068 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da34e66-633e-4f0a-88b6-5058bd71ec86 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:09.625803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da34e66-633e-4f0a-88b6-5058bd71ec86 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:09.653553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 494/972] 10.222.0.39 () {60 vars in 1141 bytes} [Thu Jul 28 22:16:09 2022] POST /identity/v3/services => generated 242 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:16:09.736243 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:09.740060 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8fbbf4-0f54-4d77-abc0-b53fc6bacd92 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:09.769317 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f045d8-3b13-4646-a4f7-2e8ee9941b9e None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:09.769632 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f045d8-3b13-4646-a4f7-2e8ee9941b9e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:09.769916 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f045d8-3b13-4646-a4f7-2e8ee9941b9e None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:09.771516 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 495/973] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:16:09.777109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66def09-15e0-486a-b92b-2e93a0371fa9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:09.777395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66def09-15e0-486a-b92b-2e93a0371fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:09.777710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66def09-15e0-486a-b92b-2e93a0371fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:09.791233 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea96b00-425c-4749-9325-9cf36db4365e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:09.791861 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea96b00-425c-4749-9325-9cf36db4365e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:09.792491 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea96b00-425c-4749-9325-9cf36db4365e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:09.792491 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea96b00-425c-4749-9325-9cf36db4365e None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:09.793095 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea96b00-425c-4749-9325-9cf36db4365e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:09.793400 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea96b00-425c-4749-9325-9cf36db4365e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:09.794803 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea96b00-425c-4749-9325-9cf36db4365e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:09.799266 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f66def09-15e0-486a-b92b-2e93a0371fa9 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:09.800476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 479/974] 10.222.0.39 () {56 vars in 1122 bytes} [Thu Jul 28 22:16:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:09.801330 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f66def09-15e0-486a-b92b-2e93a0371fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:09.807170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea96b00-425c-4749-9325-9cf36db4365e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:09.885649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 496/975] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:09 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:16:09.905873 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66def09-15e0-486a-b92b-2e93a0371fa9 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:09.912949 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea96b00-425c-4749-9325-9cf36db4365e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:09.924100 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:09.958609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b7e60-5b02-4f09-a594-7e0a4e218ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:09.959237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b7e60-5b02-4f09-a594-7e0a4e218ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:09.959809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b7e60-5b02-4f09-a594-7e0a4e218ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:09.960131 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b7e60-5b02-4f09-a594-7e0a4e218ba0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:09.960911 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b7e60-5b02-4f09-a594-7e0a4e218ba0 None admin] RBAC: Adding query filter params (interface=admin, service_id=af2dc8b463ba453790500b6ef3914c92, region_id=RegionOne) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:09.961227 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b7e60-5b02-4f09-a594-7e0a4e218ba0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:09.963072 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b7e60-5b02-4f09-a594-7e0a4e218ba0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:09.973785 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 480/976] 10.222.0.39 () {56 vars in 1249 bytes} [Thu Jul 28 22:16:09 2022] GET /identity/v3/endpoints?service_id=af2dc8b463ba453790500b6ef3914c92&interface=admin®ion_id=RegionOne => generated 193 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:09.974609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b7e60-5b02-4f09-a594-7e0a4e218ba0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:10.006492 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:10.012162 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66def09-15e0-486a-b92b-2e93a0371fa9 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:10.049372 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209697a-2199-4bb5-be62-e3b9c3a1c14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:10.050054 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209697a-2199-4bb5-be62-e3b9c3a1c14a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:10.050341 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209697a-2199-4bb5-be62-e3b9c3a1c14a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:10.050619 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0209697a-2199-4bb5-be62-e3b9c3a1c14a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:10.063061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0209697a-2199-4bb5-be62-e3b9c3a1c14a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:16:10.070134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-0209697a-2199-4bb5-be62-e3b9c3a1c14a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 22:16:10.070134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:10.070134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:10.070134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:10.070134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:10.070134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:10.070134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:10.070134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:10.070134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:10.070134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:10.070134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:10.070134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:10.070134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:16:10.070134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:16:10.070134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:16:10.070134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:16:10.071289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:16:10.071289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:16:10.071289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:16:10.071289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:16:10.071289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:10.071289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:10.071289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:10.071289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:10.071289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:10.071289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:10.071289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:10.071289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:10.071289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:10.071289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:10.071289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:10.071289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:10.072177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:10.072177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:10.072177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:10.072177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:10.072177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:16:10.072177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:16:10.072177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:16:10.072177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:16:10.072177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:16:10.072177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:16:10.072177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 22:16:10.072177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:10.073672 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 497/977] 10.222.0.39 () {56 vars in 1120 bytes} [Thu Jul 28 22:16:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:10.077370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209697a-2199-4bb5-be62-e3b9c3a1c14a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:10.093300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b7e60-5b02-4f09-a594-7e0a4e218ba0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:10.135308 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b0c4d-f509-4e94-af15-6bb1af14f556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:10.135932 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b0c4d-f509-4e94-af15-6bb1af14f556 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:10.136215 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b0c4d-f509-4e94-af15-6bb1af14f556 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:10.136741 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b0c4d-f509-4e94-af15-6bb1af14f556 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:10.137143 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9b0c4d-f509-4e94-af15-6bb1af14f556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:10.138818 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9b0c4d-f509-4e94-af15-6bb1af14f556 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:10.161892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 481/978] 10.222.0.39 () {56 vars in 1087 bytes} [Thu Jul 28 22:16:09 2022] GET /identity/v3/services => generated 1727 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:16:10.164374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:10.190121 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0209697a-2199-4bb5-be62-e3b9c3a1c14a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:10.229987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af9e8e-0035-42ef-8e4e-8d89b6b6035a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:10.230631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af9e8e-0035-42ef-8e4e-8d89b6b6035a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:10.230917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af9e8e-0035-42ef-8e4e-8d89b6b6035a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:10.231194 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4af9e8e-0035-42ef-8e4e-8d89b6b6035a None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:10.231936 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af9e8e-0035-42ef-8e4e-8d89b6b6035a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:10.232247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af9e8e-0035-42ef-8e4e-8d89b6b6035a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:10.233686 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4af9e8e-0035-42ef-8e4e-8d89b6b6035a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:10.248298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 498/979] 10.222.0.39 () {56 vars in 1115 bytes} [Thu Jul 28 22:16:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:10.249516 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b0c4d-f509-4e94-af15-6bb1af14f556 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:10.346570 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:10.360694 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b0c4d-f509-4e94-af15-6bb1af14f556 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:10.395470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a09646-e000-4ad2-9116-907c54a1df54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:10.396116 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a09646-e000-4ad2-9116-907c54a1df54 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:10.396402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a09646-e000-4ad2-9116-907c54a1df54 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:10.396680 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a09646-e000-4ad2-9116-907c54a1df54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:10.406920 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a09646-e000-4ad2-9116-907c54a1df54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:10.408370 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a09646-e000-4ad2-9116-907c54a1df54 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:10.423752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-58a09646-e000-4ad2-9116-907c54a1df54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:10.423752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:10.423752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:10.423752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:10.423752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:10.423752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:10.423752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:10.423752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:10.423752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:10.423752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:10.423752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:10.423752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:10.423752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:16:10.423752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:16:10.423752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:16:10.424779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:16:10.424779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:10.424779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:10.424779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:10.424779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:10.424779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:10.424779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:10.424779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:10.424779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:10.424779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:10.424779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:10.424779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:10.424779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:10.424779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:10.424779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:10.424779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:10.424779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:10.425747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:16:10.425747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:16:10.425747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:10.425747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:10.425747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 482/980] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:10.432219 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af9e8e-0035-42ef-8e4e-8d89b6b6035a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:10.531694 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4af9e8e-0035-42ef-8e4e-8d89b6b6035a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:10.565610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75898cfb-3cd7-431e-8d32-94343dff25c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:10.566227 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75898cfb-3cd7-431e-8d32-94343dff25c3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:10.566510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75898cfb-3cd7-431e-8d32-94343dff25c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:10.566783 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75898cfb-3cd7-431e-8d32-94343dff25c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:10.567473 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75898cfb-3cd7-431e-8d32-94343dff25c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:10.567781 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75898cfb-3cd7-431e-8d32-94343dff25c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:10.569145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75898cfb-3cd7-431e-8d32-94343dff25c3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:10.578131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:10.589004 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 499/981] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:10.590474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a09646-e000-4ad2-9116-907c54a1df54 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:10.697607 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a09646-e000-4ad2-9116-907c54a1df54 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:10.731957 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71a41c-3962-491b-8918-e9d15acad958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:10.732596 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71a41c-3962-491b-8918-e9d15acad958 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:10.732877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71a41c-3962-491b-8918-e9d15acad958 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:10.733150 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71a41c-3962-491b-8918-e9d15acad958 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:10.740691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71a41c-3962-491b-8918-e9d15acad958 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:10.742292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71a41c-3962-491b-8918-e9d15acad958 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:10.744831 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:10.754721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-2c71a41c-3962-491b-8918-e9d15acad958 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 22:16:10.754721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:10.754721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:10.754721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:10.754721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:10.754721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:10.754721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:10.754721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:10.754721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:10.754721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:10.754721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:10.754721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:10.754721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:16:10.754721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:16:10.754721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:16:10.754721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:16:10.755906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:10.755906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:10.755906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:16:10.755906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:16:10.755906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:16:10.755906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:16:10.755906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:10.755906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:10.755906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:10.755906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:10.755906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:10.755906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:10.755906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:10.755906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:10.755906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:10.755906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:10.755906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:10.755906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:10.757637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:10.757637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:10.757637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:16:10.757637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:16:10.757637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:16:10.757637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:16:10.757637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:16:10.757637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:16:10.757637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 28 22:16:10.757637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:10.760229 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 483/982] 10.222.0.39 () {56 vars in 1117 bytes} [Thu Jul 28 22:16:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:10.765763 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75898cfb-3cd7-431e-8d32-94343dff25c3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:10.870323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75898cfb-3cd7-431e-8d32-94343dff25c3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:10.904762 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dba990-cfd8-4c1b-84ad-9cea758f3cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:10.905543 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dba990-cfd8-4c1b-84ad-9cea758f3cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:10.905954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dba990-cfd8-4c1b-84ad-9cea758f3cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:10.906327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dba990-cfd8-4c1b-84ad-9cea758f3cd2 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:10.907217 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dba990-cfd8-4c1b-84ad-9cea758f3cd2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:10.907627 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dba990-cfd8-4c1b-84ad-9cea758f3cd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:10.909139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dba990-cfd8-4c1b-84ad-9cea758f3cd2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:10.931479 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 500/983] 10.222.0.39 () {56 vars in 1149 bytes} [Thu Jul 28 22:16:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:10.932870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71a41c-3962-491b-8918-e9d15acad958 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:11.032981 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71a41c-3962-491b-8918-e9d15acad958 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:11.066195 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0e814-8844-465c-92c2-4692cdba7316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:11.066958 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0e814-8844-465c-92c2-4692cdba7316 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:11.067329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0e814-8844-465c-92c2-4692cdba7316 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:11.067689 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe0e814-8844-465c-92c2-4692cdba7316 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:11.133876 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:11.137219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe0e814-8844-465c-92c2-4692cdba7316 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:11.139067 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe0e814-8844-465c-92c2-4692cdba7316 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:11.243956 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 484/984] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:16:10 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 22:16:11.249611 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dba990-cfd8-4c1b-84ad-9cea758f3cd2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:11.310687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:11.354891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dba990-cfd8-4c1b-84ad-9cea758f3cd2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:11.389107 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217b079-abda-4f94-8b00-f7bcdf2d23df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:11.389774 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217b079-abda-4f94-8b00-f7bcdf2d23df None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:11.390055 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217b079-abda-4f94-8b00-f7bcdf2d23df None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:11.390328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a217b079-abda-4f94-8b00-f7bcdf2d23df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:11.400526 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217b079-abda-4f94-8b00-f7bcdf2d23df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:11.401994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a217b079-abda-4f94-8b00-f7bcdf2d23df None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:11.417148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-a217b079-abda-4f94-8b00-f7bcdf2d23df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:11.417148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:11.417148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:11.417148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:11.417148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:11.417148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:11.417148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:11.417148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:11.417148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:11.417148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:11.417148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:11.417148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:11.417148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:16:11.417148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:16:11.417148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:16:11.418155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:16:11.418155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:11.418155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:11.418155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:11.418155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:11.418155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:11.418155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:11.418155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:11.418155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:11.418155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:11.418155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:11.418155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:11.418155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:11.418155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:11.418155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:11.418155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:11.418155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:11.426583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:16:11.426583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:16:11.426583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:11.426583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:11.426583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 501/985] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:11.426883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0e814-8844-465c-92c2-4692cdba7316 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:11.521688 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:11.525420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe0e814-8844-465c-92c2-4692cdba7316 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:11.560326 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe70972-8166-4f67-8bb3-8d3a840eb2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:11.560995 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe70972-8166-4f67-8bb3-8d3a840eb2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:11.561225 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe70972-8166-4f67-8bb3-8d3a840eb2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:11.561500 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe70972-8166-4f67-8bb3-8d3a840eb2b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:11.562218 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe70972-8166-4f67-8bb3-8d3a840eb2b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:11.562524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe70972-8166-4f67-8bb3-8d3a840eb2b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:11.563894 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe70972-8166-4f67-8bb3-8d3a840eb2b2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:11.579945 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 485/986] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:11.581512 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217b079-abda-4f94-8b00-f7bcdf2d23df None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:11.694356 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a217b079-abda-4f94-8b00-f7bcdf2d23df None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:11.705064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:11.730924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5dd771-fe96-48c5-8efd-078ee2b1dc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:11.731569 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dd771-fe96-48c5-8efd-078ee2b1dc4a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:11.731849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dd771-fe96-48c5-8efd-078ee2b1dc4a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:11.732119 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5dd771-fe96-48c5-8efd-078ee2b1dc4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:11.735810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4b9a0-4b04-4d67-b919-2ca0b1a6c4c1 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:11.736104 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4b9a0-4b04-4d67-b919-2ca0b1a6c4c1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:11.736383 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4b9a0-4b04-4d67-b919-2ca0b1a6c4c1 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:11.737936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 486/987] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:16:11.739148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5dd771-fe96-48c5-8efd-078ee2b1dc4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:11.740595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5dd771-fe96-48c5-8efd-078ee2b1dc4a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:11.743181 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029ed55-61bf-489f-bfe1-41ef31495b16 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:11.743471 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029ed55-61bf-489f-bfe1-41ef31495b16 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:11.743751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e029ed55-61bf-489f-bfe1-41ef31495b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:11.752240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-7e5dd771-fe96-48c5-8efd-078ee2b1dc4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:16:11.752240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:11.752240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:11.752240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:11.752240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:11.752240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:11.752240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:11.752240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:11.752240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:11.752240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:11.752240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:11.752240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:11.752240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:16:11.752240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:16:11.752240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:16:11.753268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:16:11.753268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:11.753268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:11.753268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:11.753268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:11.753268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:11.753268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:11.753268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:11.753268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:11.753268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:11.753268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:11.753268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:11.753268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:11.753268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:11.753268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:11.753268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:11.753268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:11.758602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:16:11.758602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:16:11.758602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:16:11.758602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:16:11.758602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:16:11.758602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:16:11.758602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:16:11.758602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:11.758602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 502/988] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:16:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:11.761224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5dd771-fe96-48c5-8efd-078ee2b1dc4a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:11.765719 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e029ed55-61bf-489f-bfe1-41ef31495b16 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:11.767012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e029ed55-61bf-489f-bfe1-41ef31495b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:11.837690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 487/989] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:11 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:11.859210 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029ed55-61bf-489f-bfe1-41ef31495b16 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:11.871153 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5dd771-fe96-48c5-8efd-078ee2b1dc4a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:11.912587 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c61ed-0d09-4ef0-a854-1adae87c1594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:11.913317 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c61ed-0d09-4ef0-a854-1adae87c1594 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:11.913639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c61ed-0d09-4ef0-a854-1adae87c1594 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:11.914514 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c61ed-0d09-4ef0-a854-1adae87c1594 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:11.915431 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c61ed-0d09-4ef0-a854-1adae87c1594 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:11.915844 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c61ed-0d09-4ef0-a854-1adae87c1594 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:11.917943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c61ed-0d09-4ef0-a854-1adae87c1594 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:11.932563 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 503/990] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:16:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:11.938443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c61ed-0d09-4ef0-a854-1adae87c1594 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:11.959135 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:11.972121 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e029ed55-61bf-489f-bfe1-41ef31495b16 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:11.984323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:12.009263 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565d3a7-2c76-452b-9fd9-b1851e8b3924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:12.009962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565d3a7-2c76-452b-9fd9-b1851e8b3924 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:12.010773 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565d3a7-2c76-452b-9fd9-b1851e8b3924 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:12.011055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565d3a7-2c76-452b-9fd9-b1851e8b3924 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:12.011684 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565d3a7-2c76-452b-9fd9-b1851e8b3924 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:12.013067 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565d3a7-2c76-452b-9fd9-b1851e8b3924 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:12.023133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-4565d3a7-2c76-452b-9fd9-b1851e8b3924 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 22:16:12.023133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:12.023133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:16:12.023133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:16:12.023133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:16:12.023133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:16:12.023133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:16:12.023133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:12.023133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 22:16:12.023133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:12.023133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:16:12.023133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:12.023133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:12.023133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:12.023133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:12.023133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:12.023133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:12.024174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:12.024174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:12.024174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:12.024174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:12.024174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:12.024174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:12.024174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:16:12.024174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:16:12.024174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:16:12.024174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:16:12.024174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:12.024174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:12.024174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:12.024174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:12.024174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:12.024174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:12.025068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:12.025068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:12.025068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:12.025068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:12.025068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:12.025068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:12.025068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:12.025068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:12.025068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:12.025068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:12.025068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:16:12.025068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:12.025068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 22:16:12.025068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:12.025068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 488/991] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:16:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:12.030175 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565d3a7-2c76-452b-9fd9-b1851e8b3924 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:12.062238 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c61ed-0d09-4ef0-a854-1adae87c1594 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:12.098089 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b4768d-0375-4156-b422-30d6c84281d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:12.098708 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4768d-0375-4156-b422-30d6c84281d0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:12.098991 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4768d-0375-4156-b422-30d6c84281d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:12.099262 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b4768d-0375-4156-b422-30d6c84281d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:12.100051 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4768d-0375-4156-b422-30d6c84281d0 None admin] RBAC: Adding query filter params (role.id=62bb6d1764134942a1069bf3a5c1e0c4, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=85626350279c40ea99da93b9ed6994b2) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:12.100360 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4768d-0375-4156-b422-30d6c84281d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:12.102081 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b4768d-0375-4156-b422-30d6c84281d0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:12.108231 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 504/992] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:16:11 2022] GET /identity/v3/role_assignments?user.id=85626350279c40ea99da93b9ed6994b2&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=62bb6d1764134942a1069bf3a5c1e0c4 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:12.162490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565d3a7-2c76-452b-9fd9-b1851e8b3924 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:12.177130 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25139711-9b9e-43b3-af24-07b77a88db56 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:12.177576 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25139711-9b9e-43b3-af24-07b77a88db56 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:12.177865 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25139711-9b9e-43b3-af24-07b77a88db56 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:12.179650 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 505/993] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:16:12.185181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210d733-a3e7-47c3-8e4d-dd50cba92500 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:12.185471 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210d733-a3e7-47c3-8e4d-dd50cba92500 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:12.185778 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2210d733-a3e7-47c3-8e4d-dd50cba92500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:12.201732 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6fce5-698e-4524-8854-389805e2aef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:12.202365 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6fce5-698e-4524-8854-389805e2aef5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:12.202679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6fce5-698e-4524-8854-389805e2aef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:12.202920 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6fce5-698e-4524-8854-389805e2aef5 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:12.203613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6fce5-698e-4524-8854-389805e2aef5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:12.203918 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6fce5-698e-4524-8854-389805e2aef5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:12.205307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f6fce5-698e-4524-8854-389805e2aef5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:12.206022 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-2210d733-a3e7-47c3-8e4d-dd50cba92500 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:12.207590 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2210d733-a3e7-47c3-8e4d-dd50cba92500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:12.211690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 489/994] 10.222.0.39 () {56 vars in 1112 bytes} [Thu Jul 28 22:16:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:12.217499 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6fce5-698e-4524-8854-389805e2aef5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:12.239801 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:12.290469 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 506/995] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:12 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:16:12.313922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210d733-a3e7-47c3-8e4d-dd50cba92500 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:12.334427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6fce5-698e-4524-8854-389805e2aef5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:12.343115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:12.372372 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ff50a-2dbf-4df5-86b3-021c3b649388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:12.374013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ff50a-2dbf-4df5-86b3-021c3b649388 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:12.374303 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ff50a-2dbf-4df5-86b3-021c3b649388 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:12.374584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25ff50a-2dbf-4df5-86b3-021c3b649388 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:12.375282 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ff50a-2dbf-4df5-86b3-021c3b649388 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:12.375593 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ff50a-2dbf-4df5-86b3-021c3b649388 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:12.376970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25ff50a-2dbf-4df5-86b3-021c3b649388 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:12.384387 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 490/996] 10.222.0.39 () {56 vars in 1112 bytes} [Thu Jul 28 22:16:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:12.390504 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ff50a-2dbf-4df5-86b3-021c3b649388 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:12.431818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2210d733-a3e7-47c3-8e4d-dd50cba92500 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:12.469083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b85cfb-5d58-4736-b53c-b85f38d3f885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:12.469758 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b85cfb-5d58-4736-b53c-b85f38d3f885 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:12.470043 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b85cfb-5d58-4736-b53c-b85f38d3f885 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:12.470725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b85cfb-5d58-4736-b53c-b85f38d3f885 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:12.471365 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b85cfb-5d58-4736-b53c-b85f38d3f885 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:12.473114 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b85cfb-5d58-4736-b53c-b85f38d3f885 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:12.477825 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:12.483987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-31b85cfb-5d58-4736-b53c-b85f38d3f885 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 22:16:12.483987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:12.483987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:16:12.483987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:16:12.483987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:16:12.483987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:16:12.483987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:16:12.483987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:12.483987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 22:16:12.483987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:12.483987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:16:12.483987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:12.483987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:12.483987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:12.483987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:12.483987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:12.483987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:12.485015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:12.485015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:12.485015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:12.485015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:12.485015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:12.485015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:12.485015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:16:12.485015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:16:12.485015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:16:12.485015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:16:12.485015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:12.485015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:12.485015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:12.485015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:12.485015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:12.485015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:12.485926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:12.485926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:12.485926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:12.485926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:12.485926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:12.485926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:12.485926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:12.485926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:12.485926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:12.485926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:12.485926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:16:12.485926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:12.485926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 28 22:16:12.485926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:12.485926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 507/997] 10.222.0.39 () {56 vars in 1126 bytes} [Thu Jul 28 22:16:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:12.490970 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b85cfb-5d58-4736-b53c-b85f38d3f885 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:12.503900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25ff50a-2dbf-4df5-86b3-021c3b649388 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:12.543301 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac72fe-0867-4720-a68b-a3b4f99aec13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:12.543927 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac72fe-0867-4720-a68b-a3b4f99aec13 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:12.544212 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac72fe-0867-4720-a68b-a3b4f99aec13 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:12.544489 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac72fe-0867-4720-a68b-a3b4f99aec13 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:12.545244 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac72fe-0867-4720-a68b-a3b4f99aec13 None admin] RBAC: Adding query filter params (interface=public, service_id=c95af8f792f9475f9d2253a6694941a2, region_id=RegionOne) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:12.545577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac72fe-0867-4720-a68b-a3b4f99aec13 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:12.546949 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ac72fe-0867-4720-a68b-a3b4f99aec13 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:12.552729 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 491/998] 10.222.0.39 () {56 vars in 1251 bytes} [Thu Jul 28 22:16:12 2022] GET /identity/v3/endpoints?service_id=c95af8f792f9475f9d2253a6694941a2&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:12.558710 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac72fe-0867-4720-a68b-a3b4f99aec13 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:12.596260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:12.611399 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b85cfb-5d58-4736-b53c-b85f38d3f885 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:12.650525 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1246fa2-316e-4428-b0a4-a08c65749f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:12.651461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1246fa2-316e-4428-b0a4-a08c65749f28 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:12.652045 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1246fa2-316e-4428-b0a4-a08c65749f28 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:12.652539 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1246fa2-316e-4428-b0a4-a08c65749f28 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:12.653459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1246fa2-316e-4428-b0a4-a08c65749f28 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:12.653942 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1246fa2-316e-4428-b0a4-a08c65749f28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:12.655566 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1246fa2-316e-4428-b0a4-a08c65749f28 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:12.662651 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 508/999] 10.222.0.39 () {56 vars in 1122 bytes} [Thu Jul 28 22:16:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:12.668912 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1246fa2-316e-4428-b0a4-a08c65749f28 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:12.678087 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac72fe-0867-4720-a68b-a3b4f99aec13 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:12.720251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d00242-29bc-4d44-a624-45a50c6632e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:12.720970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d00242-29bc-4d44-a624-45a50c6632e7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:12.721253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d00242-29bc-4d44-a624-45a50c6632e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:12.721551 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d00242-29bc-4d44-a624-45a50c6632e7 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:12.722209 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d00242-29bc-4d44-a624-45a50c6632e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:12.723643 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d00242-29bc-4d44-a624-45a50c6632e7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:12.741831 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 492/1000] 10.222.0.39 () {56 vars in 1087 bytes} [Thu Jul 28 22:16:12 2022] GET /identity/v3/services => generated 1727 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:16:12.779661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1246fa2-316e-4428-b0a4-a08c65749f28 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:12.818405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5953f-c555-4554-96ea-f5e55ff5a004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:12.819031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5953f-c555-4554-96ea-f5e55ff5a004 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:12.819386 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5953f-c555-4554-96ea-f5e55ff5a004 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:12.819711 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5953f-c555-4554-96ea-f5e55ff5a004 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:12.820665 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5953f-c555-4554-96ea-f5e55ff5a004 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:12.820969 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5953f-c555-4554-96ea-f5e55ff5a004 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:12.822392 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5953f-c555-4554-96ea-f5e55ff5a004 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:12.832946 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 509/1001] 10.222.0.39 () {56 vars in 1122 bytes} [Thu Jul 28 22:16:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:12.833797 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d00242-29bc-4d44-a624-45a50c6632e7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:12.847668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:12.942140 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d00242-29bc-4d44-a624-45a50c6632e7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:12.978155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15568dd6-a60d-4738-bf95-a87e17ca47fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:12.979167 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15568dd6-a60d-4738-bf95-a87e17ca47fd None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:12.979597 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15568dd6-a60d-4738-bf95-a87e17ca47fd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:12.980046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15568dd6-a60d-4738-bf95-a87e17ca47fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:12.981408 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15568dd6-a60d-4738-bf95-a87e17ca47fd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:12.982964 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15568dd6-a60d-4738-bf95-a87e17ca47fd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:13.000881 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 493/1002] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:16:12 2022] POST /identity/v3/endpoints => generated 325 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:16:14.191209 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1d179-a74b-420d-acdc-1951b979f5a8 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:14.191710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1d179-a74b-420d-acdc-1951b979f5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:14.191787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1d179-a74b-420d-acdc-1951b979f5a8 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:14.193279 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 510/1003] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:14.199000 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3ba2e-3498-4f1f-aecf-d70748a7bf45 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:14.199381 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3ba2e-3498-4f1f-aecf-d70748a7bf45 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:14.199559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3ba2e-3498-4f1f-aecf-d70748a7bf45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:14.217998 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4fb3ba2e-3498-4f1f-aecf-d70748a7bf45 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:14.219193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4fb3ba2e-3498-4f1f-aecf-d70748a7bf45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:14.284651 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:14.298832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 494/1004] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:14 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:14.321092 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1d179-a74b-420d-acdc-1951b979f5a8 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:14.340659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:14.428996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1d179-a74b-420d-acdc-1951b979f5a8 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:14.464845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66178cd-7713-4181-a611-5fbb740a59b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:14.465487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66178cd-7713-4181-a611-5fbb740a59b7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:14.465796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66178cd-7713-4181-a611-5fbb740a59b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:14.466075 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66178cd-7713-4181-a611-5fbb740a59b7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:14.466771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66178cd-7713-4181-a611-5fbb740a59b7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:14.468212 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66178cd-7713-4181-a611-5fbb740a59b7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:14.477185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-e66178cd-7713-4181-a611-5fbb740a59b7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 22:16:14.477185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:14.477185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:16:14.477185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:16:14.477185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:16:14.477185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:16:14.477185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:16:14.477185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:14.477185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 22:16:14.477185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:14.477185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:16:14.477185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:14.477185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:14.477185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:14.477185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:14.477185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:14.477185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:14.478222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:14.478222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:14.478222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:14.478222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:14.478222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:14.478222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:14.478222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:16:14.478222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:16:14.478222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:16:14.478222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:16:14.478222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:14.478222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:14.478222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:14.478222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:14.478222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:14.478222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:14.479073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:14.479073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:14.479073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:14.479073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:14.479073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:14.479073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:14.479073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:14.479073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:14.479073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:14.479073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:14.479073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:16:14.479073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:14.479073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 22:16:14.479073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:14.481848 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 511/1005] 10.222.0.39 () {56 vars in 1105 bytes} [Thu Jul 28 22:16:14 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:14.484461 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3ba2e-3498-4f1f-aecf-d70748a7bf45 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:14.590797 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3ba2e-3498-4f1f-aecf-d70748a7bf45 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:14.628138 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1649f-de4e-4443-8540-601e1a67af30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:14.628758 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1649f-de4e-4443-8540-601e1a67af30 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:14.629041 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1649f-de4e-4443-8540-601e1a67af30 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:14.629313 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1649f-de4e-4443-8540-601e1a67af30 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:14.630035 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1649f-de4e-4443-8540-601e1a67af30 None admin] RBAC: Adding query filter params (name=image) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:14.630341 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1649f-de4e-4443-8540-601e1a67af30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:14.631702 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1649f-de4e-4443-8540-601e1a67af30 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:14.642136 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 495/1006] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:14.643401 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66178cd-7713-4181-a611-5fbb740a59b7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:14.729600 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:14.750790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66178cd-7713-4181-a611-5fbb740a59b7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:14.752202 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ec75b-9906-4afc-9b29-70fa53068c97 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:14.752491 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ec75b-9906-4afc-9b29-70fa53068c97 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:14.752766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ec75b-9906-4afc-9b29-70fa53068c97 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:14.754273 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 496/1007] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:16:14.759091 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbe77a-fe39-4e19-abe7-d52ca456c613 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:14.759373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbe77a-fe39-4e19-abe7-d52ca456c613 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:14.759642 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbe77a-fe39-4e19-abe7-d52ca456c613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:14.778195 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e9bbe77a-fe39-4e19-abe7-d52ca456c613 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:14.779716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e9bbe77a-fe39-4e19-abe7-d52ca456c613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:14.787841 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00741ae3-77c9-4e2a-bacf-b09151d696b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:14.789191 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00741ae3-77c9-4e2a-bacf-b09151d696b9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:14.789489 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00741ae3-77c9-4e2a-bacf-b09151d696b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:14.789762 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00741ae3-77c9-4e2a-bacf-b09151d696b9 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:14.790442 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00741ae3-77c9-4e2a-bacf-b09151d696b9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:14.790747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00741ae3-77c9-4e2a-bacf-b09151d696b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:14.792112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00741ae3-77c9-4e2a-bacf-b09151d696b9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:14.796934 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 512/1008] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:14 2022] GET /identity/v3/services?type=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:14.843778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:14.857559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 497/1009] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:14 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:16:14.880719 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00741ae3-77c9-4e2a-bacf-b09151d696b9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:14.989712 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00741ae3-77c9-4e2a-bacf-b09151d696b9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:15.025576 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8bdae-c88a-4557-a363-763bde4251d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:15.026595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8bdae-c88a-4557-a363-763bde4251d6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:15.027101 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8bdae-c88a-4557-a363-763bde4251d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:15.027601 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c8bdae-c88a-4557-a363-763bde4251d6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:15.028424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8bdae-c88a-4557-a363-763bde4251d6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:15.030034 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c8bdae-c88a-4557-a363-763bde4251d6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:15.037608 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00462891-7ef7-40a2-bf08-e940d90d93f9 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:15.038439 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00462891-7ef7-40a2-bf08-e940d90d93f9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:15.038715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00462891-7ef7-40a2-bf08-e940d90d93f9 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:15.040171 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 498/1010] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:16:15.040964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-10c8bdae-c88a-4557-a363-763bde4251d6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 22:16:15.040964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:15.040964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:16:15.040964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:16:15.040964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:16:15.040964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:16:15.040964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:16:15.040964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:15.040964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 22:16:15.040964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:15.040964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:16:15.040964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:15.040964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:15.040964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:15.040964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:15.040964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:15.040964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:15.041975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:15.041975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:15.041975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:15.041975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:15.041975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:15.041975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:15.041975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:16:15.041975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:16:15.041975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:16:15.041975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:16:15.041975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:15.041975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:15.041975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:15.041975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:15.041975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:15.041975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:15.042829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:15.042829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:15.042829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:15.042829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:15.042829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:15.042829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:15.042829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:15.042829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:15.042829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:15.042829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:15.042829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:16:15.042829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:15.042829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 28 22:16:15.042829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:15.042829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 513/1011] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:16:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:15.045145 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1c3ea-ff5d-4df1-b5e3-ba30b82be39f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:15.045430 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1c3ea-ff5d-4df1-b5e3-ba30b82be39f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:15.045739 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c1c3ea-ff5d-4df1-b5e3-ba30b82be39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:15.049184 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8bdae-c88a-4557-a363-763bde4251d6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:15.060943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:15.067465 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a2c1c3ea-ff5d-4df1-b5e3-ba30b82be39f None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:15.068662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a2c1c3ea-ff5d-4df1-b5e3-ba30b82be39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:15.145572 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 499/1012] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:15 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:16:15.168629 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c8bdae-c88a-4557-a363-763bde4251d6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:15.169644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1c3ea-ff5d-4df1-b5e3-ba30b82be39f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:15.207092 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0555a3-fa23-4961-9b4d-88cf863b9777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:15.207721 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0555a3-fa23-4961-9b4d-88cf863b9777 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:15.208004 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0555a3-fa23-4961-9b4d-88cf863b9777 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:15.208282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0555a3-fa23-4961-9b4d-88cf863b9777 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:15.208986 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0555a3-fa23-4961-9b4d-88cf863b9777 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:15.209389 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0555a3-fa23-4961-9b4d-88cf863b9777 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:15.210810 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0555a3-fa23-4961-9b4d-88cf863b9777 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:15.215821 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 514/1013] 10.222.0.39 () {56 vars in 1112 bytes} [Thu Jul 28 22:16:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:15.221365 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0555a3-fa23-4961-9b4d-88cf863b9777 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:15.284981 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c1c3ea-ff5d-4df1-b5e3-ba30b82be39f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:15.298256 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:15.307987 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:15.323827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af337582-6d82-48d1-adfa-ed35197a1224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:15.324490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af337582-6d82-48d1-adfa-ed35197a1224 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:15.324779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af337582-6d82-48d1-adfa-ed35197a1224 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:15.325059 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af337582-6d82-48d1-adfa-ed35197a1224 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:15.326266 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af337582-6d82-48d1-adfa-ed35197a1224 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:15.327710 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af337582-6d82-48d1-adfa-ed35197a1224 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:15.329462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 500/1014] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:15.335696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af337582-6d82-48d1-adfa-ed35197a1224 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:15.335856 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0555a3-fa23-4961-9b4d-88cf863b9777 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:15.369983 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05fc73-a0b4-43ef-98a0-36d3f3ef281b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:15.370605 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05fc73-a0b4-43ef-98a0-36d3f3ef281b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:15.370892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05fc73-a0b4-43ef-98a0-36d3f3ef281b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:15.371175 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05fc73-a0b4-43ef-98a0-36d3f3ef281b None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:15.371861 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05fc73-a0b4-43ef-98a0-36d3f3ef281b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:15.372171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05fc73-a0b4-43ef-98a0-36d3f3ef281b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:15.373557 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05fc73-a0b4-43ef-98a0-36d3f3ef281b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:15.378538 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 515/1015] 10.222.0.39 () {56 vars in 1112 bytes} [Thu Jul 28 22:16:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:15.384103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05fc73-a0b4-43ef-98a0-36d3f3ef281b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:15.452233 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af337582-6d82-48d1-adfa-ed35197a1224 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:15.487876 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b84d67-10bb-4d23-8317-04f7b1b2375d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:15.488503 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b84d67-10bb-4d23-8317-04f7b1b2375d None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:15.488788 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b84d67-10bb-4d23-8317-04f7b1b2375d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:15.489064 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b84d67-10bb-4d23-8317-04f7b1b2375d None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:15.489777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b84d67-10bb-4d23-8317-04f7b1b2375d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:15.491728 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b84d67-10bb-4d23-8317-04f7b1b2375d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:15.494456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05fc73-a0b4-43ef-98a0-36d3f3ef281b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:15.521754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 501/1016] 10.222.0.39 () {60 vars in 1141 bytes} [Thu Jul 28 22:16:15 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:16:15.530989 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95465da-1990-4db2-a28f-6055ae69070b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:15.531635 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95465da-1990-4db2-a28f-6055ae69070b None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:15.531919 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95465da-1990-4db2-a28f-6055ae69070b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:15.532194 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95465da-1990-4db2-a28f-6055ae69070b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:15.532932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95465da-1990-4db2-a28f-6055ae69070b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:15.534393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95465da-1990-4db2-a28f-6055ae69070b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:15.541676 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:15.561972 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 516/1017] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:16:15 2022] POST /identity/v3/endpoints => generated 334 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:16:16.827815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde9da0-d88c-4bad-841b-4e81783f9cd1 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:16.828340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde9da0-d88c-4bad-841b-4e81783f9cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:16.828419 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde9da0-d88c-4bad-841b-4e81783f9cd1 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:16.829795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 502/1018] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:16.834977 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71a057-6e2f-473e-a4a9-b2ba51dce4b9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:16.835455 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71a057-6e2f-473e-a4a9-b2ba51dce4b9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:16.835532 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d71a057-6e2f-473e-a4a9-b2ba51dce4b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:16.854072 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6d71a057-6e2f-473e-a4a9-b2ba51dce4b9 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:16.855271 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6d71a057-6e2f-473e-a4a9-b2ba51dce4b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:16.929557 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 517/1019] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:16 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:16.951948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde9da0-d88c-4bad-841b-4e81783f9cd1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:16.995666 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:17.059808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde9da0-d88c-4bad-841b-4e81783f9cd1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:17.094411 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5894281a-6c7f-4da4-b210-e455d7da67ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:17.095035 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894281a-6c7f-4da4-b210-e455d7da67ae None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:17.095320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894281a-6c7f-4da4-b210-e455d7da67ae None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:17.095596 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5894281a-6c7f-4da4-b210-e455d7da67ae None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:17.096302 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894281a-6c7f-4da4-b210-e455d7da67ae None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:17.097700 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5894281a-6c7f-4da4-b210-e455d7da67ae None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:17.121570 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 503/1020] 10.222.0.39 () {60 vars in 1141 bytes} [Thu Jul 28 22:16:16 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:16:17.577721 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f227a2-c064-4309-99fe-6e6c5820e6ff None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:17.578065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f227a2-c064-4309-99fe-6e6c5820e6ff None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:17.578179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f227a2-c064-4309-99fe-6e6c5820e6ff None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:17.579596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 518/1021] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:17.584824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c7df3-df44-4451-960b-3244a8ca01f3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:17.585255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c7df3-df44-4451-960b-3244a8ca01f3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:17.585601 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c7df3-df44-4451-960b-3244a8ca01f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:17.604245 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9f7c7df3-df44-4451-960b-3244a8ca01f3 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:17.605422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9f7c7df3-df44-4451-960b-3244a8ca01f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:17.680355 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 504/1022] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:17.704437 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f227a2-c064-4309-99fe-6e6c5820e6ff None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:17.729198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e69588-c91d-4279-a7e3-f4327ab429e9 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:17.729597 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e69588-c91d-4279-a7e3-f4327ab429e9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:17.729804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e69588-c91d-4279-a7e3-f4327ab429e9 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:17.731146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 505/1023] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:16:17.736180 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded1d2c-49d5-431f-80ba-5657e8924d6a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:17.736461 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded1d2c-49d5-431f-80ba-5657e8924d6a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:17.736731 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded1d2c-49d5-431f-80ba-5657e8924d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:17.759722 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-dded1d2c-49d5-431f-80ba-5657e8924d6a None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:17.760916 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-dded1d2c-49d5-431f-80ba-5657e8924d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:17.782880 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:17.821934 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f227a2-c064-4309-99fe-6e6c5820e6ff None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:17.836364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 506/1024] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:16:17.840938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:17.860602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded1d2c-49d5-431f-80ba-5657e8924d6a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:17.861209 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fcb5e9-72f3-437b-8b75-8a454c24a9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:17.861894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcb5e9-72f3-437b-8b75-8a454c24a9ad None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:17.862182 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcb5e9-72f3-437b-8b75-8a454c24a9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:17.862453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fcb5e9-72f3-437b-8b75-8a454c24a9ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:17.863623 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fcb5e9-72f3-437b-8b75-8a454c24a9ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:17.865057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fcb5e9-72f3-437b-8b75-8a454c24a9ad None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:17.871876 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 519/1025] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:17.872783 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fcb5e9-72f3-437b-8b75-8a454c24a9ad None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:17.972605 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded1d2c-49d5-431f-80ba-5657e8924d6a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:17.979351 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fcb5e9-72f3-437b-8b75-8a454c24a9ad None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:18.011826 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b05d3a-0c2a-4d96-a57e-cad055ddc25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:18.012467 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b05d3a-0c2a-4d96-a57e-cad055ddc25b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:18.012748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b05d3a-0c2a-4d96-a57e-cad055ddc25b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:18.013020 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b05d3a-0c2a-4d96-a57e-cad055ddc25b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:18.019500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaded84-2bab-4ab0-9dc2-0dee8b71a3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:18.020202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaded84-2bab-4ab0-9dc2-0dee8b71a3a8 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:18.020405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaded84-2bab-4ab0-9dc2-0dee8b71a3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:18.020685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eaded84-2bab-4ab0-9dc2-0dee8b71a3a8 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:18.021513 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaded84-2bab-4ab0-9dc2-0dee8b71a3a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:18.022931 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eaded84-2bab-4ab0-9dc2-0dee8b71a3a8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:18.024891 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b05d3a-0c2a-4d96-a57e-cad055ddc25b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:16:18.029841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a3b05d3a-0c2a-4d96-a57e-cad055ddc25b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 22:16:18.029841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:18.029841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:18.029841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:18.029841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:18.029841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:18.029841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:18.029841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:18.029841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:18.029841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:18.029841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:18.029841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:18.029841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:16:18.029841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:16:18.029841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:16:18.029841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:16:18.030746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:16:18.030746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:16:18.030746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:16:18.030746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:16:18.030746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:18.030746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:18.030746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:18.030746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:18.030746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:18.030746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:18.030746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:18.030746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:18.030746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:18.030746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:18.030746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:18.030746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:18.031618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:18.031618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:18.031618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:18.031618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:18.031618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:16:18.031618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:16:18.031618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:16:18.031618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:16:18.031618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:16:18.031618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:16:18.031618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 22:16:18.031618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:18.031618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 507/1026] 10.222.0.39 () {56 vars in 1120 bytes} [Thu Jul 28 22:16:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:18.037677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b05d3a-0c2a-4d96-a57e-cad055ddc25b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:18.057556 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 520/1027] 10.222.0.39 () {60 vars in 1132 bytes} [Thu Jul 28 22:16:17 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:16:18.067375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:18.182170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b05d3a-0c2a-4d96-a57e-cad055ddc25b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:18.221354 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5a66c-7262-4f46-bc66-dd4ef018f8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:18.222284 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5a66c-7262-4f46-bc66-dd4ef018f8be None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:18.222751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5a66c-7262-4f46-bc66-dd4ef018f8be None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:18.223026 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca5a66c-7262-4f46-bc66-dd4ef018f8be None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:18.223760 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5a66c-7262-4f46-bc66-dd4ef018f8be None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:18.224067 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5a66c-7262-4f46-bc66-dd4ef018f8be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:18.226152 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca5a66c-7262-4f46-bc66-dd4ef018f8be None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:18.235672 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 508/1028] 10.222.0.39 () {56 vars in 1115 bytes} [Thu Jul 28 22:16:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:18.241576 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaded84-2bab-4ab0-9dc2-0dee8b71a3a8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:18.292259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:18.354663 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eaded84-2bab-4ab0-9dc2-0dee8b71a3a8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:18.389937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae663438-0ae2-42a1-a4b1-a040f65f1626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:18.390582 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae663438-0ae2-42a1-a4b1-a040f65f1626 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:18.390868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae663438-0ae2-42a1-a4b1-a040f65f1626 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:18.391147 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae663438-0ae2-42a1-a4b1-a040f65f1626 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:18.401869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae663438-0ae2-42a1-a4b1-a040f65f1626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:18.403316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae663438-0ae2-42a1-a4b1-a040f65f1626 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:18.418451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ae663438-0ae2-42a1-a4b1-a040f65f1626 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:18.418451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:18.418451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:18.418451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:18.418451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:18.418451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:18.418451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:18.418451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:18.418451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:18.418451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:18.418451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:18.418451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:18.418451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:16:18.418451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:16:18.418451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:16:18.419726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:16:18.419726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:18.419726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:18.419726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:18.419726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:18.419726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:18.419726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:18.419726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:18.419726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:18.419726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:18.419726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:18.419726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:18.419726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:18.419726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:18.419726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:18.419726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:18.419726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:18.420937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:16:18.420937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:16:18.420937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:18.420937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:18.420937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 521/1029] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:18.427871 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5a66c-7262-4f46-bc66-dd4ef018f8be None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:18.510423 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:18.533973 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca5a66c-7262-4f46-bc66-dd4ef018f8be None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:18.569116 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8377f-98a4-4738-b62d-1a56117a7b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:18.570070 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8377f-98a4-4738-b62d-1a56117a7b39 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:18.570538 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8377f-98a4-4738-b62d-1a56117a7b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:18.570995 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8377f-98a4-4738-b62d-1a56117a7b39 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:18.571905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8377f-98a4-4738-b62d-1a56117a7b39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:18.572394 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8377f-98a4-4738-b62d-1a56117a7b39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:18.573975 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd8377f-98a4-4738-b62d-1a56117a7b39 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:18.587712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 509/1030] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:18.592888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae663438-0ae2-42a1-a4b1-a040f65f1626 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:18.691203 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:18.699695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae663438-0ae2-42a1-a4b1-a040f65f1626 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:18.735596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2d549-62da-4aae-8308-2d188dc45db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:18.736243 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2d549-62da-4aae-8308-2d188dc45db4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:18.736521 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2d549-62da-4aae-8308-2d188dc45db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:18.736790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2d549-62da-4aae-8308-2d188dc45db4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:18.745230 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e2d549-62da-4aae-8308-2d188dc45db4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:18.746708 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e2d549-62da-4aae-8308-2d188dc45db4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:18.758913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-b0e2d549-62da-4aae-8308-2d188dc45db4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 22:16:18.758913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:18.758913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:18.758913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:18.758913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:18.758913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:18.758913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:18.758913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:18.758913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:18.758913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:18.758913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:18.758913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:18.758913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:16:18.758913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:16:18.758913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:16:18.758913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:16:18.759929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:18.759929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:18.759929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:16:18.759929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:16:18.759929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:16:18.759929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:16:18.759929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:18.759929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:18.759929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:18.759929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:18.759929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:18.759929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:18.759929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:18.759929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:18.759929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:18.759929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:18.759929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:18.759929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:18.760888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:18.760888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:18.760888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:16:18.760888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:16:18.760888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:16:18.760888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:16:18.760888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:16:18.760888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:16:18.760888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 22:16:18.760888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:18.762268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 522/1031] 10.222.0.39 () {56 vars in 1093 bytes} [Thu Jul 28 22:16:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:18.765961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8377f-98a4-4738-b62d-1a56117a7b39 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:18.873392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd8377f-98a4-4738-b62d-1a56117a7b39 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:18.908218 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b68448-8b86-4911-bedb-cf25cabddcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:18.908835 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b68448-8b86-4911-bedb-cf25cabddcb7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:18.909113 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b68448-8b86-4911-bedb-cf25cabddcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:18.909383 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b68448-8b86-4911-bedb-cf25cabddcb7 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:18.910214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b68448-8b86-4911-bedb-cf25cabddcb7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:18.910518 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b68448-8b86-4911-bedb-cf25cabddcb7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:18.911936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b68448-8b86-4911-bedb-cf25cabddcb7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:18.920313 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:18.932160 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 510/1032] 10.222.0.39 () {56 vars in 1133 bytes} [Thu Jul 28 22:16:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:18.938166 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2d549-62da-4aae-8308-2d188dc45db4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:19.043430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2d549-62da-4aae-8308-2d188dc45db4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:19.078107 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cbb1d-eafa-4ce0-9543-bb97a0f8a218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:19.078726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cbb1d-eafa-4ce0-9543-bb97a0f8a218 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:19.079006 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cbb1d-eafa-4ce0-9543-bb97a0f8a218 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:19.079281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82cbb1d-eafa-4ce0-9543-bb97a0f8a218 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:19.104773 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:19.153442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e44b7-0b2b-400d-a0e7-fd2a3e3d4d73 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:19.154066 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e44b7-0b2b-400d-a0e7-fd2a3e3d4d73 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:19.154408 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e44b7-0b2b-400d-a0e7-fd2a3e3d4d73 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:19.156157 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 511/1033] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:16:19.156618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82cbb1d-eafa-4ce0-9543-bb97a0f8a218 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:19.158124 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82cbb1d-eafa-4ce0-9543-bb97a0f8a218 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:19.163617 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22456-7b8e-4249-a738-1e2397f5575f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:19.163936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22456-7b8e-4249-a738-1e2397f5575f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:19.164208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb22456-7b8e-4249-a738-1e2397f5575f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:19.182533 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9cb22456-7b8e-4249-a738-1e2397f5575f None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:19.183707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9cb22456-7b8e-4249-a738-1e2397f5575f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:19.257481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 512/1034] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:16:19.268585 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 523/1035] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:16:18 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 22:16:19.274271 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cbb1d-eafa-4ce0-9543-bb97a0f8a218 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:19.281028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb22456-7b8e-4249-a738-1e2397f5575f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:19.338489 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:19.386218 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb22456-7b8e-4249-a738-1e2397f5575f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:19.390501 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82cbb1d-eafa-4ce0-9543-bb97a0f8a218 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:19.417171 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:19.426670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83610a16-2215-437a-b191-0fcceb539eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:19.427305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83610a16-2215-437a-b191-0fcceb539eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:19.427565 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853f916-262b-43f6-af03-0ab829564412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:19.427682 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83610a16-2215-437a-b191-0fcceb539eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:19.427878 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83610a16-2215-437a-b191-0fcceb539eb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:19.428251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853f916-262b-43f6-af03-0ab829564412 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:19.429159 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853f916-262b-43f6-af03-0ab829564412 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:19.429435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853f916-262b-43f6-af03-0ab829564412 None admin] PATH_INFO: `/v3/services/image` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:19.430103 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5853f916-262b-43f6-af03-0ab829564412 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:19.431965 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5853f916-262b-43f6-af03-0ab829564412 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:19.438661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83610a16-2215-437a-b191-0fcceb539eb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:19.440086 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83610a16-2215-437a-b191-0fcceb539eb1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:19.441724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-5853f916-262b-43f6-af03-0ab829564412 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 22:16:19.441724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:19.441724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:16:19.441724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:16:19.441724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:16:19.441724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:16:19.441724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:16:19.441724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:19.441724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 22:16:19.441724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:19.441724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:16:19.441724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:19.441724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:19.441724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:19.441724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:19.441724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:19.441724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:19.442919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:19.442919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:19.442919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:19.442919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:19.442919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:19.442919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:19.442919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:16:19.442919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:16:19.442919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:16:19.442919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:16:19.442919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:19.442919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:19.442919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:19.442919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:19.442919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:19.442919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:19.443794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:19.443794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:19.443794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:19.443794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:19.443794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:19.443794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:19.443794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:19.443794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:19.443794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:19.443794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:19.443794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:16:19.443794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:19.443794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 22:16:19.443794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:19.443794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 513/1036] 10.222.0.39 () {56 vars in 1105 bytes} [Thu Jul 28 22:16:19 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:19.448709 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853f916-262b-43f6-af03-0ab829564412 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:19.459025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-83610a16-2215-437a-b191-0fcceb539eb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:19.459025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:19.459025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:19.459025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:19.459025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:19.459025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:19.459025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:19.459025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:19.459025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:19.459025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:19.459025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:19.459025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:19.459025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:16:19.459025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:16:19.459025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:16:19.460482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:16:19.460482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:19.460482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:19.460482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:19.460482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:19.460482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:19.460482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:19.460482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:19.460482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:19.460482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:19.460482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:19.460482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:19.460482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:19.460482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:19.460482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:19.460482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:19.460482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:19.461377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:16:19.461377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:16:19.461377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:19.461377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:19.461660 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 524/1037] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:19.466974 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83610a16-2215-437a-b191-0fcceb539eb1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:19.553127 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853f916-262b-43f6-af03-0ab829564412 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:19.584001 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83610a16-2215-437a-b191-0fcceb539eb1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:19.591827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:19.592198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1440edb-8753-4151-b5bc-8b4e24433212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:19.592820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1440edb-8753-4151-b5bc-8b4e24433212 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:19.593258 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1440edb-8753-4151-b5bc-8b4e24433212 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:19.593366 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1440edb-8753-4151-b5bc-8b4e24433212 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:19.594071 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1440edb-8753-4151-b5bc-8b4e24433212 None admin] RBAC: Adding query filter params (name=image) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:19.594424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1440edb-8753-4151-b5bc-8b4e24433212 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:19.596114 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1440edb-8753-4151-b5bc-8b4e24433212 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:19.608105 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 514/1038] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:19.609220 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1440edb-8753-4151-b5bc-8b4e24433212 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:19.621310 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd8c50-1e1f-43fb-9408-3426f9f848d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:19.621981 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd8c50-1e1f-43fb-9408-3426f9f848d8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:19.622266 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd8c50-1e1f-43fb-9408-3426f9f848d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:19.622544 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd8c50-1e1f-43fb-9408-3426f9f848d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:19.623241 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd8c50-1e1f-43fb-9408-3426f9f848d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:19.623549 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd8c50-1e1f-43fb-9408-3426f9f848d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:19.624924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fd8c50-1e1f-43fb-9408-3426f9f848d8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:19.637834 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 525/1039] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:19.643717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd8c50-1e1f-43fb-9408-3426f9f848d8 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:19.668181 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:19.716773 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1440edb-8753-4151-b5bc-8b4e24433212 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:19.749907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd8c50-1e1f-43fb-9408-3426f9f848d8 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:19.753878 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84634bcb-2766-4010-8685-e11b1249b478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:19.754515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84634bcb-2766-4010-8685-e11b1249b478 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:19.754793 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84634bcb-2766-4010-8685-e11b1249b478 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:19.755071 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84634bcb-2766-4010-8685-e11b1249b478 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:19.755763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84634bcb-2766-4010-8685-e11b1249b478 None admin] RBAC: Adding query filter params (type=image) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:19.756070 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84634bcb-2766-4010-8685-e11b1249b478 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:19.758102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84634bcb-2766-4010-8685-e11b1249b478 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:19.768820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 515/1040] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:19.769803 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84634bcb-2766-4010-8685-e11b1249b478 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:19.795345 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0da98-2974-4c9e-bddc-e72d4f087c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:19.795982 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0da98-2974-4c9e-bddc-e72d4f087c5c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:19.796268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0da98-2974-4c9e-bddc-e72d4f087c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:19.796545 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b0da98-2974-4c9e-bddc-e72d4f087c5c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:19.803566 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b0da98-2974-4c9e-bddc-e72d4f087c5c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:19.805121 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b0da98-2974-4c9e-bddc-e72d4f087c5c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:19.817964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-29b0da98-2974-4c9e-bddc-e72d4f087c5c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:16:19.817964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:19.817964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:19.817964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:19.817964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:19.817964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:19.817964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:19.817964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:19.817964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:19.817964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:19.817964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:19.817964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:19.817964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:16:19.817964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:16:19.817964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:16:19.819809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:16:19.819809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:19.819809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:19.819809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:19.819809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:19.819809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:19.819809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:19.819809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:19.819809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:19.819809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:19.819809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:19.819809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:19.819809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:19.819809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:19.819809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:19.819809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:19.819809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:19.820747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:16:19.820747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:16:19.820747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:16:19.820747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:16:19.820747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:16:19.820747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:16:19.820747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:16:19.820747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:19.820747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 526/1041] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:16:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:19.825475 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0da98-2974-4c9e-bddc-e72d4f087c5c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:19.854949 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:19.887021 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84634bcb-2766-4010-8685-e11b1249b478 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:19.906944 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:19.926228 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4fd20-8f64-4168-b02c-00c4f4695614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:19.926848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4fd20-8f64-4168-b02c-00c4f4695614 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:19.927126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4fd20-8f64-4168-b02c-00c4f4695614 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:19.927398 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4fd20-8f64-4168-b02c-00c4f4695614 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:19.928144 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4fd20-8f64-4168-b02c-00c4f4695614 None admin] RBAC: Adding query filter params (interface=public, service_id=20ddabaf6f3b4832a2baa342f66576dc, region_id=RegionOne) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:19.928728 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4fd20-8f64-4168-b02c-00c4f4695614 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:19.930132 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f4fd20-8f64-4168-b02c-00c4f4695614 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:19.935219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 516/1042] 10.222.0.39 () {56 vars in 1251 bytes} [Thu Jul 28 22:16:19 2022] GET /identity/v3/endpoints?service_id=20ddabaf6f3b4832a2baa342f66576dc&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:19.935632 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b0da98-2974-4c9e-bddc-e72d4f087c5c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:19.940684 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4fd20-8f64-4168-b02c-00c4f4695614 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:19.973382 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee90e0-f47f-426c-928e-370127995f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:19.974028 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee90e0-f47f-426c-928e-370127995f98 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:19.974307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee90e0-f47f-426c-928e-370127995f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:19.974580 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bee90e0-f47f-426c-928e-370127995f98 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:19.975355 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee90e0-f47f-426c-928e-370127995f98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:19.975932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee90e0-f47f-426c-928e-370127995f98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:19.977120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bee90e0-f47f-426c-928e-370127995f98 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:19.989542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 527/1043] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:16:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:19.995535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee90e0-f47f-426c-928e-370127995f98 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:20.058655 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f4fd20-8f64-4168-b02c-00c4f4695614 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:20.091301 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:20.098284 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eed35e-e748-48ea-bd7f-bd9236156fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:20.098911 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eed35e-e748-48ea-bd7f-bd9236156fff None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:20.099192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eed35e-e748-48ea-bd7f-bd9236156fff None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:20.099475 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eed35e-e748-48ea-bd7f-bd9236156fff None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:20.100112 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eed35e-e748-48ea-bd7f-bd9236156fff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:20.102292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eed35e-e748-48ea-bd7f-bd9236156fff None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:20.108852 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 517/1044] 10.222.0.39 () {56 vars in 1087 bytes} [Thu Jul 28 22:16:19 2022] GET /identity/v3/services => generated 1957 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:16:20.109059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bee90e0-f47f-426c-928e-370127995f98 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:20.135751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a979b-2f26-4be8-840c-98bb560d1363 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:20.136111 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a979b-2f26-4be8-840c-98bb560d1363 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:20.136318 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151a979b-2f26-4be8-840c-98bb560d1363 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:20.138820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 518/1045] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:16:20.145674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46720c56-f632-458f-947a-9e622c6ba335 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:20.146011 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46720c56-f632-458f-947a-9e622c6ba335 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:20.146234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46720c56-f632-458f-947a-9e622c6ba335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:20.158571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d361f-05fb-4d13-9299-5cfe5a6e9348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:20.159207 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d361f-05fb-4d13-9299-5cfe5a6e9348 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:20.159487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d361f-05fb-4d13-9299-5cfe5a6e9348 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:20.159759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d361f-05fb-4d13-9299-5cfe5a6e9348 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:20.160552 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d361f-05fb-4d13-9299-5cfe5a6e9348 None admin] RBAC: Adding query filter params (role.id=62bb6d1764134942a1069bf3a5c1e0c4, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=77efe8046ea0468cb0c7c58c24fc4e75) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:20.160861 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d361f-05fb-4d13-9299-5cfe5a6e9348 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:20.162361 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d361f-05fb-4d13-9299-5cfe5a6e9348 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:20.166786 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-46720c56-f632-458f-947a-9e622c6ba335 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:20.168096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-46720c56-f632-458f-947a-9e622c6ba335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:20.169102 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 528/1046] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:16:19 2022] GET /identity/v3/role_assignments?user.id=77efe8046ea0468cb0c7c58c24fc4e75&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=62bb6d1764134942a1069bf3a5c1e0c4 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:20.222277 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:20.253148 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 519/1047] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:16:20.278329 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d361f-05fb-4d13-9299-5cfe5a6e9348 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:20.395622 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d361f-05fb-4d13-9299-5cfe5a6e9348 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:20.431249 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77181470-625c-45c3-a2b5-85d771c4a04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:20.431897 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77181470-625c-45c3-a2b5-85d771c4a04f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:20.432180 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77181470-625c-45c3-a2b5-85d771c4a04f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:20.432457 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77181470-625c-45c3-a2b5-85d771c4a04f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:20.443176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77181470-625c-45c3-a2b5-85d771c4a04f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:16:20.448973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-77181470-625c-45c3-a2b5-85d771c4a04f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:16:20.448973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:20.448973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:20.448973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:20.448973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:20.448973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:20.448973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:20.448973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:20.448973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:20.448973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:20.448973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:20.448973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:20.448973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:16:20.448973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:16:20.448973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:16:20.448973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:16:20.449932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:16:20.449932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:16:20.449932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:16:20.449932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:16:20.449932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:20.449932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:20.449932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:20.449932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:20.449932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:20.449932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:20.449932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:20.449932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:20.449932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:20.449932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:20.449932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:20.449932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:20.450824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:20.450824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:20.450824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:20.450824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:20.450824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:16:20.450824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:16:20.450824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:16:20.450824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:16:20.450824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:16:20.450824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:16:20.450824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:16:20.450824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:20.450824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 529/1048] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:16:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:20.456081 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46720c56-f632-458f-947a-9e622c6ba335 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:20.562281 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46720c56-f632-458f-947a-9e622c6ba335 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:20.597093 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d6222-b9c2-49bc-982a-eff10e5f7b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:20.597739 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d6222-b9c2-49bc-982a-eff10e5f7b85 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:20.598027 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d6222-b9c2-49bc-982a-eff10e5f7b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:20.598302 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d6222-b9c2-49bc-982a-eff10e5f7b85 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:20.599027 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d6222-b9c2-49bc-982a-eff10e5f7b85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:20.599333 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d6222-b9c2-49bc-982a-eff10e5f7b85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:20.601319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5d6222-b9c2-49bc-982a-eff10e5f7b85 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:20.616417 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 520/1049] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:16:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:20.618092 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77181470-625c-45c3-a2b5-85d771c4a04f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:20.630892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:20.726096 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77181470-625c-45c3-a2b5-85d771c4a04f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:20.760625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27fa60-df80-4d18-9419-fcedd9ef833f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:20.761268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27fa60-df80-4d18-9419-fcedd9ef833f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:20.761564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27fa60-df80-4d18-9419-fcedd9ef833f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:20.761841 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27fa60-df80-4d18-9419-fcedd9ef833f None admin] PATH_INFO: `/v3/users/55c6cdb859e147989561153f24a2dd78` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:20.775770 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27fa60-df80-4d18-9419-fcedd9ef833f None admin] RBAC: Authorizing `identity:get_user(user_id=55c6cdb859e147989561153f24a2dd78)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:20.777200 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27fa60-df80-4d18-9419-fcedd9ef833f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:20.783562 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 530/1050] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:16:20 2022] GET /identity/v3/users/55c6cdb859e147989561153f24a2dd78 => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:20.785319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d6222-b9c2-49bc-982a-eff10e5f7b85 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:20.827914 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:20.891215 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d6222-b9c2-49bc-982a-eff10e5f7b85 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:20.924913 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18632ee1-b227-4677-a4fd-bd1271fe3345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:20.925544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18632ee1-b227-4677-a4fd-bd1271fe3345 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:20.925832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18632ee1-b227-4677-a4fd-bd1271fe3345 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:20.926102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18632ee1-b227-4677-a4fd-bd1271fe3345 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:20.993380 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18632ee1-b227-4677-a4fd-bd1271fe3345 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:20.994769 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18632ee1-b227-4677-a4fd-bd1271fe3345 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:21.036952 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:21.105031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 521/1051] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:16:20 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 22:16:21.106686 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27fa60-df80-4d18-9419-fcedd9ef833f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:21.182494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:21.217619 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27fa60-df80-4d18-9419-fcedd9ef833f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:21.253581 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaaa6cf-7a1d-4605-83a8-eb2a6a7724d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:21.254135 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaaa6cf-7a1d-4605-83a8-eb2a6a7724d8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:21.254417 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaaa6cf-7a1d-4605-83a8-eb2a6a7724d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:21.254692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaaa6cf-7a1d-4605-83a8-eb2a6a7724d8 None admin] PATH_INFO: `/v3/projects/ccb474c36db842eea7c3658d041eb982` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:21.255849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaaa6cf-7a1d-4605-83a8-eb2a6a7724d8 None admin] RBAC: Authorizing `identity:get_project(project_id=ccb474c36db842eea7c3658d041eb982)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:21.257936 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaaa6cf-7a1d-4605-83a8-eb2a6a7724d8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:21.264725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 531/1052] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:16:21 2022] GET /identity/v3/projects/ccb474c36db842eea7c3658d041eb982 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:21.266307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18632ee1-b227-4677-a4fd-bd1271fe3345 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:21.374920 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18632ee1-b227-4677-a4fd-bd1271fe3345 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:21.409436 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57976ae4-19a6-4397-b138-68f23027c8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:21.410073 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57976ae4-19a6-4397-b138-68f23027c8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:21.410356 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57976ae4-19a6-4397-b138-68f23027c8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:21.410631 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57976ae4-19a6-4397-b138-68f23027c8c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:21.411420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57976ae4-19a6-4397-b138-68f23027c8c8 None admin] RBAC: Adding query filter params (role.id=aded7bd43e974686b3c23bd9d4255b24, scope.project.id=ccb474c36db842eea7c3658d041eb982, user.id=55c6cdb859e147989561153f24a2dd78) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:21.411732 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57976ae4-19a6-4397-b138-68f23027c8c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:21.413719 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57976ae4-19a6-4397-b138-68f23027c8c8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:21.419241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 522/1053] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:16:21 2022] GET /identity/v3/role_assignments?user.id=55c6cdb859e147989561153f24a2dd78&scope.project.id=ccb474c36db842eea7c3658d041eb982&role.id=aded7bd43e974686b3c23bd9d4255b24 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:22.189863 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d965350-0ee1-405d-9e62-ae21b0e7718d None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:22.190337 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d965350-0ee1-405d-9e62-ae21b0e7718d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:22.190414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d965350-0ee1-405d-9e62-ae21b0e7718d None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:22.191898 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 532/1054] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:22.196780 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff29bb7-8c1d-41d1-af6f-0c085630f00d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:22.197198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff29bb7-8c1d-41d1-af6f-0c085630f00d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:22.197377 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff29bb7-8c1d-41d1-af6f-0c085630f00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:22.203577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:22.216967 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e078bd-dc76-4414-8aea-7431d7895a8d None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:22.217138 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-aff29bb7-8c1d-41d1-af6f-0c085630f00d None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:22.217242 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e078bd-dc76-4414-8aea-7431d7895a8d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:22.217582 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e078bd-dc76-4414-8aea-7431d7895a8d None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:22.218244 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-aff29bb7-8c1d-41d1-af6f-0c085630f00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:22.219018 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 533/1055] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:16:22.223626 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a7c50-1411-417f-abff-cd75a0a43d90 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:22.223921 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a7c50-1411-417f-abff-cd75a0a43d90 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:22.224204 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a7c50-1411-417f-abff-cd75a0a43d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:22.242904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f23a7c50-1411-417f-abff-cd75a0a43d90 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:22.244114 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f23a7c50-1411-417f-abff-cd75a0a43d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:22.297801 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 523/1056] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:22.312930 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff29bb7-8c1d-41d1-af6f-0c085630f00d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:22.314079 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 534/1057] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:16:22.336612 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a7c50-1411-417f-abff-cd75a0a43d90 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:22.412271 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:22.426765 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff29bb7-8c1d-41d1-af6f-0c085630f00d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:22.449321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a7c50-1411-417f-abff-cd75a0a43d90 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:22.467458 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:22.474334 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031af1f-5f27-4030-8bba-5ee92f63cdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:22.474987 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031af1f-5f27-4030-8bba-5ee92f63cdcc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:22.475296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031af1f-5f27-4030-8bba-5ee92f63cdcc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:22.475567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031af1f-5f27-4030-8bba-5ee92f63cdcc None admin] PATH_INFO: `/v3/services/image` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:22.476203 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031af1f-5f27-4030-8bba-5ee92f63cdcc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:22.477598 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031af1f-5f27-4030-8bba-5ee92f63cdcc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:22.487550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-6031af1f-5f27-4030-8bba-5ee92f63cdcc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 22:16:22.487550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:22.487550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:16:22.487550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:16:22.487550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:16:22.487550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:16:22.487550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:16:22.487550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:22.487550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 22:16:22.487550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:22.487550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:16:22.487550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:22.487550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:22.487550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:22.487550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:22.487550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:22.487550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:22.488589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:22.488589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:22.488589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:22.488589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:22.488589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:22.488589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:22.488589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:16:22.488589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:16:22.488589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:16:22.488589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:16:22.488589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:22.488589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:22.488589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:22.488589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:22.488589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:22.488589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:22.489905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:22.489905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:22.489905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:22.489905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:22.489905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:22.489905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:22.489905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:22.489905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:22.489905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:22.489905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:22.489905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:16:22.489905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:16:22.489905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 28 22:16:22.489905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:22.489905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 524/1058] 10.222.0.39 () {56 vars in 1105 bytes} [Thu Jul 28 22:16:22 2022] GET /identity/v3/services/image => generated 86 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:22.495137 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031af1f-5f27-4030-8bba-5ee92f63cdcc None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:22.499403 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f626701-6ebe-40d3-9fce-c8a2149cf588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:22.500347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f626701-6ebe-40d3-9fce-c8a2149cf588 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:22.500637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f626701-6ebe-40d3-9fce-c8a2149cf588 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:22.500916 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f626701-6ebe-40d3-9fce-c8a2149cf588 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:22.515653 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f626701-6ebe-40d3-9fce-c8a2149cf588 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:16:22.521984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-8f626701-6ebe-40d3-9fce-c8a2149cf588 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 22:16:22.521984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:22.521984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:22.521984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:22.521984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:22.521984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:22.521984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:22.521984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:22.521984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:22.521984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:22.521984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:22.521984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:22.521984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:16:22.521984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:16:22.521984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:16:22.521984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:16:22.522956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:16:22.522956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:16:22.522956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:16:22.522956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:16:22.522956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:22.522956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:22.522956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:22.522956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:22.522956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:22.522956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:22.522956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:22.522956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:22.522956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:22.522956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:22.522956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:22.522956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:22.523835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:22.523835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:22.523835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:22.523835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:22.523835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:16:22.523835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:16:22.523835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:16:22.523835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:16:22.523835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:16:22.523835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:16:22.523835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 22:16:22.523835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:22.524490 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 535/1059] 10.222.0.39 () {56 vars in 1120 bytes} [Thu Jul 28 22:16:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:22.529968 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f626701-6ebe-40d3-9fce-c8a2149cf588 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:22.610700 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031af1f-5f27-4030-8bba-5ee92f63cdcc None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:22.641402 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f626701-6ebe-40d3-9fce-c8a2149cf588 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:22.647588 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a92a6-8a29-4f73-be96-5bf2d4a451a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:22.648211 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a92a6-8a29-4f73-be96-5bf2d4a451a1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:22.648492 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a92a6-8a29-4f73-be96-5bf2d4a451a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:22.648766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a92a6-8a29-4f73-be96-5bf2d4a451a1 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:22.649449 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a92a6-8a29-4f73-be96-5bf2d4a451a1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:22.649784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a92a6-8a29-4f73-be96-5bf2d4a451a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:22.651151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a92a6-8a29-4f73-be96-5bf2d4a451a1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:22.656028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 525/1060] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:22.661771 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a92a6-8a29-4f73-be96-5bf2d4a451a1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:22.672487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:22.684075 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893d87a-4ead-4b17-b621-bc85202c00a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:22.684715 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893d87a-4ead-4b17-b621-bc85202c00a5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:22.684980 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893d87a-4ead-4b17-b621-bc85202c00a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:22.685255 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c893d87a-4ead-4b17-b621-bc85202c00a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:22.686018 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893d87a-4ead-4b17-b621-bc85202c00a5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:22.686323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893d87a-4ead-4b17-b621-bc85202c00a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:22.687684 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c893d87a-4ead-4b17-b621-bc85202c00a5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:22.696808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 536/1061] 10.222.0.39 () {56 vars in 1115 bytes} [Thu Jul 28 22:16:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:22.702331 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893d87a-4ead-4b17-b621-bc85202c00a5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:22.730471 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:22.775053 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a92a6-8a29-4f73-be96-5bf2d4a451a1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:22.812364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbe5a9-3435-46a4-9cc7-2f9d2edcacae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:22.813387 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbe5a9-3435-46a4-9cc7-2f9d2edcacae None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:22.814281 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbe5a9-3435-46a4-9cc7-2f9d2edcacae None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:22.814565 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fbe5a9-3435-46a4-9cc7-2f9d2edcacae None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:22.815252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbe5a9-3435-46a4-9cc7-2f9d2edcacae None admin] RBAC: Adding query filter params (type=image) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:22.815588 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbe5a9-3435-46a4-9cc7-2f9d2edcacae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:22.816966 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fbe5a9-3435-46a4-9cc7-2f9d2edcacae None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:22.817896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c893d87a-4ead-4b17-b621-bc85202c00a5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:22.827873 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 526/1062] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:22.829670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbe5a9-3435-46a4-9cc7-2f9d2edcacae None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:22.855853 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881e443-47c8-4946-ba9d-3a42ea177085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:22.856547 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881e443-47c8-4946-ba9d-3a42ea177085 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:22.856840 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881e443-47c8-4946-ba9d-3a42ea177085 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:22.857122 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4881e443-47c8-4946-ba9d-3a42ea177085 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:22.868482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881e443-47c8-4946-ba9d-3a42ea177085 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:22.869963 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4881e443-47c8-4946-ba9d-3a42ea177085 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:22.884864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-4881e443-47c8-4946-ba9d-3a42ea177085 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:22.884864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:22.884864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:22.884864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:22.884864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:22.884864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:22.884864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:22.884864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:22.884864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:22.884864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:22.884864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:22.884864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:22.884864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:16:22.884864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:16:22.884864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:16:22.886062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:16:22.886062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:22.886062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:22.886062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:22.886062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:22.886062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:22.886062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:22.886062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:22.886062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:22.886062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:22.886062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:22.886062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:22.886062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:22.886062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:22.886062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:22.886062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:22.886062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:22.887005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:16:22.887005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:16:22.887005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:22.887005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:22.887005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 537/1063] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:22.891899 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881e443-47c8-4946-ba9d-3a42ea177085 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:22.937057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:22.943457 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fbe5a9-3435-46a4-9cc7-2f9d2edcacae None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:22.969341 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:22.983370 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75017ed8-4b78-4184-b50c-89c4fb19c27a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:22.983681 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75017ed8-4b78-4184-b50c-89c4fb19c27a None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:22.983968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75017ed8-4b78-4184-b50c-89c4fb19c27a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:22.984247 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75017ed8-4b78-4184-b50c-89c4fb19c27a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:22.984959 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75017ed8-4b78-4184-b50c-89c4fb19c27a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:22.986362 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75017ed8-4b78-4184-b50c-89c4fb19c27a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:23.004176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4881e443-47c8-4946-ba9d-3a42ea177085 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:23.015470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 527/1064] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:16:22 2022] POST /identity/v3/endpoints => generated 327 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:16:23.050959 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39818e5-6849-44f2-8777-ef494c66e817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:23.051598 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39818e5-6849-44f2-8777-ef494c66e817 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:23.051883 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39818e5-6849-44f2-8777-ef494c66e817 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:23.052159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39818e5-6849-44f2-8777-ef494c66e817 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:23.052896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39818e5-6849-44f2-8777-ef494c66e817 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:23.054132 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39818e5-6849-44f2-8777-ef494c66e817 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:23.055572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39818e5-6849-44f2-8777-ef494c66e817 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:23.070187 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 538/1065] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:23.076057 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75017ed8-4b78-4184-b50c-89c4fb19c27a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:23.198344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75017ed8-4b78-4184-b50c-89c4fb19c27a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:23.233293 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e96d97-da62-4ef8-8573-9ccdf3bfa244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:23.233970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e96d97-da62-4ef8-8573-9ccdf3bfa244 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:23.234253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e96d97-da62-4ef8-8573-9ccdf3bfa244 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:23.234529 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e96d97-da62-4ef8-8573-9ccdf3bfa244 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:23.242986 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e96d97-da62-4ef8-8573-9ccdf3bfa244 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:23.244432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e96d97-da62-4ef8-8573-9ccdf3bfa244 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:23.255970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-01e96d97-da62-4ef8-8573-9ccdf3bfa244 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 22:16:23.255970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:23.255970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:23.255970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:23.255970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:23.255970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:23.255970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:23.255970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:23.255970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:23.255970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:23.255970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:23.255970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:23.255970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:16:23.255970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:16:23.255970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:16:23.255970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:16:23.257032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:23.257032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:23.257032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:16:23.257032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:16:23.257032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:16:23.257032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:16:23.257032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:23.257032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:23.257032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:23.257032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:23.257032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:23.257032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:23.257032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:23.257032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:23.257032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:23.257032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:23.257032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:23.257032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:23.258099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:23.258099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:23.258099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:16:23.258099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:16:23.258099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:16:23.258099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:16:23.258099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:16:23.258099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:16:23.258099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 22:16:23.258099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:23.258099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 528/1066] 10.222.0.39 () {56 vars in 1093 bytes} [Thu Jul 28 22:16:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:23.264918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39818e5-6849-44f2-8777-ef494c66e817 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:23.363707 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:23.372148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39818e5-6849-44f2-8777-ef494c66e817 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:23.407473 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1029ff2f-61e7-486e-b4cf-a51efe23f8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:23.408088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029ff2f-61e7-486e-b4cf-a51efe23f8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:23.408375 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029ff2f-61e7-486e-b4cf-a51efe23f8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:23.408655 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1029ff2f-61e7-486e-b4cf-a51efe23f8e1 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:23.409457 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029ff2f-61e7-486e-b4cf-a51efe23f8e1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:23.409799 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029ff2f-61e7-486e-b4cf-a51efe23f8e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:23.411246 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1029ff2f-61e7-486e-b4cf-a51efe23f8e1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:23.430878 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 539/1067] 10.222.0.39 () {56 vars in 1133 bytes} [Thu Jul 28 22:16:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:23.436295 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e96d97-da62-4ef8-8573-9ccdf3bfa244 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:23.450365 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867480cd-e57f-4fe3-8ffd-559e19d3bb8b None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:23.450982 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867480cd-e57f-4fe3-8ffd-559e19d3bb8b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:23.451483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867480cd-e57f-4fe3-8ffd-559e19d3bb8b None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:23.453055 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 540/1068] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:16:23.458964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862e82a-8233-41f4-b098-ae1b84b8bdea None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:23.459532 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862e82a-8233-41f4-b098-ae1b84b8bdea None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:23.460018 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862e82a-8233-41f4-b098-ae1b84b8bdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:23.481207 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8862e82a-8233-41f4-b098-ae1b84b8bdea None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:23.482449 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8862e82a-8233-41f4-b098-ae1b84b8bdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:23.482859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:23.548944 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e96d97-da62-4ef8-8573-9ccdf3bfa244 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:23.556138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 541/1069] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:16:23.578861 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862e82a-8233-41f4-b098-ae1b84b8bdea None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:23.587319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399842a6-e61a-4063-b7ce-c05b97738e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:23.588759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399842a6-e61a-4063-b7ce-c05b97738e71 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:23.589244 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399842a6-e61a-4063-b7ce-c05b97738e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:23.590147 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399842a6-e61a-4063-b7ce-c05b97738e71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:23.603691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399842a6-e61a-4063-b7ce-c05b97738e71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:23.605160 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399842a6-e61a-4063-b7ce-c05b97738e71 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:23.620558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-399842a6-e61a-4063-b7ce-c05b97738e71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:23.620558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:23.620558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:23.620558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:23.620558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:23.620558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:23.620558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:23.620558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:23.620558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:23.620558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:23.620558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:23.620558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:23.620558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:16:23.620558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:16:23.620558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:16:23.621468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:16:23.621468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:23.621468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:23.621468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:23.621468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:23.621468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:23.621468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:23.621468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:23.621468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:23.621468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:23.621468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:23.621468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:23.621468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:23.621468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:23.621468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:23.621468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:23.621468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:23.622458 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:16:23.622458 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:16:23.622458 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:23.622458 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:23.625863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 529/1070] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:23.627264 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399842a6-e61a-4063-b7ce-c05b97738e71 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:23.672692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:23.690154 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8862e82a-8233-41f4-b098-ae1b84b8bdea None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:23.727639 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:23.729804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03568ea-2d0a-4f6f-82d9-a52d5c12d5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:23.730979 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03568ea-2d0a-4f6f-82d9-a52d5c12d5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:23.731444 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03568ea-2d0a-4f6f-82d9-a52d5c12d5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:23.731718 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03568ea-2d0a-4f6f-82d9-a52d5c12d5b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:23.737001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399842a6-e61a-4063-b7ce-c05b97738e71 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:23.744191 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03568ea-2d0a-4f6f-82d9-a52d5c12d5b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:16:23.749818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-b03568ea-2d0a-4f6f-82d9-a52d5c12d5b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:16:23.749818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:23.749818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:23.749818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:23.749818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:23.749818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:23.749818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:23.749818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:23.749818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:23.749818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:23.749818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:23.749818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:23.749818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:16:23.749818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:16:23.749818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:16:23.749818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:16:23.750759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:16:23.750759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:16:23.750759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:16:23.750759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:16:23.750759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:23.750759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:23.750759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:23.750759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:23.750759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:23.750759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:23.750759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:23.750759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:23.750759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:23.750759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:23.750759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:23.750759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:23.751606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:23.751606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:23.751606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:23.751606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:23.751606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:16:23.751606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:16:23.751606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:16:23.751606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:16:23.751606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:16:23.751606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:16:23.751606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:16:23.751606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:23.751606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 542/1071] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:16:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:23.757904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03568ea-2d0a-4f6f-82d9-a52d5c12d5b9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:23.778285 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd477fd-8487-473f-b9ca-22d80c4f51b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:23.779312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd477fd-8487-473f-b9ca-22d80c4f51b1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:23.779594 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd477fd-8487-473f-b9ca-22d80c4f51b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:23.779865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd477fd-8487-473f-b9ca-22d80c4f51b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:23.780970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd477fd-8487-473f-b9ca-22d80c4f51b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:23.781441 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd477fd-8487-473f-b9ca-22d80c4f51b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:23.783462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd477fd-8487-473f-b9ca-22d80c4f51b1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:23.801298 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 530/1072] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:23.802413 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd477fd-8487-473f-b9ca-22d80c4f51b1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:23.869658 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03568ea-2d0a-4f6f-82d9-a52d5c12d5b9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:23.904565 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be1fae-fdde-4468-acfe-ca364a19500b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:23.905382 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be1fae-fdde-4468-acfe-ca364a19500b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:23.905838 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be1fae-fdde-4468-acfe-ca364a19500b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:23.906265 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8be1fae-fdde-4468-acfe-ca364a19500b None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:23.906975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be1fae-fdde-4468-acfe-ca364a19500b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:23.907283 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be1fae-fdde-4468-acfe-ca364a19500b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:23.908695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8be1fae-fdde-4468-acfe-ca364a19500b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:23.915397 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd477fd-8487-473f-b9ca-22d80c4f51b1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:23.921938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 543/1073] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:16:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:23.922798 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be1fae-fdde-4468-acfe-ca364a19500b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:23.931794 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:23.953227 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6ae9d-8ad9-42e7-974e-ca0d52aa1253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:23.953910 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6ae9d-8ad9-42e7-974e-ca0d52aa1253 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:23.954194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6ae9d-8ad9-42e7-974e-ca0d52aa1253 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:23.954737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f6ae9d-8ad9-42e7-974e-ca0d52aa1253 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:23.962242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6ae9d-8ad9-42e7-974e-ca0d52aa1253 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:23.963691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f6ae9d-8ad9-42e7-974e-ca0d52aa1253 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:23.976123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-46f6ae9d-8ad9-42e7-974e-ca0d52aa1253 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:16:23.976123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:23.976123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:23.976123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:23.976123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:23.976123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:23.976123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:23.976123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:23.976123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:23.976123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:23.976123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:23.976123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:23.976123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:16:23.976123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:16:23.976123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:16:23.977013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:16:23.977013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:23.977013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:23.977013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:23.977013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:23.977013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:23.977013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:23.977013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:23.977013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:23.977013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:23.977013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:23.977013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:23.977013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:23.977013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:23.977013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:23.977013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:23.977013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:23.978092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:16:23.978092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:16:23.978092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:16:23.978092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:16:23.978092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:16:23.978092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:16:23.978092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:16:23.978092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:23.983481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 531/1074] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:16:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:23.984626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6ae9d-8ad9-42e7-974e-ca0d52aa1253 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:24.000305 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:24.036008 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8be1fae-fdde-4468-acfe-ca364a19500b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:24.081115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4930ee1-b1c8-4330-82f9-f8a0c3ae188a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:24.081878 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4930ee1-b1c8-4330-82f9-f8a0c3ae188a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:24.082226 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4930ee1-b1c8-4330-82f9-f8a0c3ae188a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:24.082567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4930ee1-b1c8-4330-82f9-f8a0c3ae188a None admin] PATH_INFO: `/v3/users/55c6cdb859e147989561153f24a2dd78` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:24.083822 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4930ee1-b1c8-4330-82f9-f8a0c3ae188a None admin] RBAC: Authorizing `identity:get_user(user_id=55c6cdb859e147989561153f24a2dd78)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:24.085715 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4930ee1-b1c8-4330-82f9-f8a0c3ae188a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:24.092239 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 544/1075] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:16:23 2022] GET /identity/v3/users/55c6cdb859e147989561153f24a2dd78 => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:24.093025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4930ee1-b1c8-4330-82f9-f8a0c3ae188a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:24.103176 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f6ae9d-8ad9-42e7-974e-ca0d52aa1253 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:24.150455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d0c5c-5c41-4f9c-bdc3-77d43327257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:24.151086 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d0c5c-5c41-4f9c-bdc3-77d43327257d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:24.151370 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d0c5c-5c41-4f9c-bdc3-77d43327257d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:24.151670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41d0c5c-5c41-4f9c-bdc3-77d43327257d None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:24.152405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d0c5c-5c41-4f9c-bdc3-77d43327257d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:24.152744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d0c5c-5c41-4f9c-bdc3-77d43327257d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:24.154205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41d0c5c-5c41-4f9c-bdc3-77d43327257d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:24.172726 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 532/1076] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:16:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:24.174963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d0c5c-5c41-4f9c-bdc3-77d43327257d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:24.191313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:24.218390 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4930ee1-b1c8-4330-82f9-f8a0c3ae188a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:24.254162 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:24.256784 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a3f4b-a382-4e26-944a-217fa4974132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:24.257519 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a3f4b-a382-4e26-944a-217fa4974132 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:24.257867 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a3f4b-a382-4e26-944a-217fa4974132 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:24.258227 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a3f4b-a382-4e26-944a-217fa4974132 None admin] PATH_INFO: `/v3/projects/ccb474c36db842eea7c3658d041eb982` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:24.259348 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a3f4b-a382-4e26-944a-217fa4974132 None admin] RBAC: Authorizing `identity:get_project(project_id=ccb474c36db842eea7c3658d041eb982)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:24.261273 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a3f4b-a382-4e26-944a-217fa4974132 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:24.268531 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 545/1077] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:16:24 2022] GET /identity/v3/projects/ccb474c36db842eea7c3658d041eb982 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:24.270256 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a3f4b-a382-4e26-944a-217fa4974132 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:24.292886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41d0c5c-5c41-4f9c-bdc3-77d43327257d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:24.330906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e544b2-3b1b-4782-8e33-1d6f22282e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:24.331734 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e544b2-3b1b-4782-8e33-1d6f22282e99 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:24.332018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e544b2-3b1b-4782-8e33-1d6f22282e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:24.332296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e544b2-3b1b-4782-8e33-1d6f22282e99 None admin] PATH_INFO: `/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/77efe8046ea0468cb0c7c58c24fc4e75/roles/62bb6d1764134942a1069bf3a5c1e0c4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:24.335192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e544b2-3b1b-4782-8e33-1d6f22282e99 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user_id=77efe8046ea0468cb0c7c58c24fc4e75, role_id=62bb6d1764134942a1069bf3a5c1e0c4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:24.336648 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e544b2-3b1b-4782-8e33-1d6f22282e99 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:24.365605 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 533/1078] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:16:24 2022] PUT /identity/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/77efe8046ea0468cb0c7c58c24fc4e75/roles/62bb6d1764134942a1069bf3a5c1e0c4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:16:24.384461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a3f4b-a382-4e26-944a-217fa4974132 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:24.425233 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e968e-a47e-41ba-afb3-92443df1cf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:24.426073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e968e-a47e-41ba-afb3-92443df1cf64 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:24.426362 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e968e-a47e-41ba-afb3-92443df1cf64 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:24.426641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e968e-a47e-41ba-afb3-92443df1cf64 None admin] PATH_INFO: `/v3/projects/ccb474c36db842eea7c3658d041eb982/users/55c6cdb859e147989561153f24a2dd78/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:24.429582 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e968e-a47e-41ba-afb3-92443df1cf64 None admin] RBAC: Authorizing `identity:create_grant(project_id=ccb474c36db842eea7c3658d041eb982, user_id=55c6cdb859e147989561153f24a2dd78, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:24.431112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924e968e-a47e-41ba-afb3-92443df1cf64 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:24.457567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 546/1079] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:16:24 2022] PUT /identity/v3/projects/ccb474c36db842eea7c3658d041eb982/users/55c6cdb859e147989561153f24a2dd78/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:16:25.080733 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c2334-b9a8-43b3-9c86-bd5161c37cc9 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:25.081118 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c2334-b9a8-43b3-9c86-bd5161c37cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:25.081298 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878c2334-b9a8-43b3-9c86-bd5161c37cc9 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:25.082799 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 534/1080] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:25.088414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280057c7-1257-4626-a022-85c30acf9872 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:25.088706 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280057c7-1257-4626-a022-85c30acf9872 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:25.088984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280057c7-1257-4626-a022-85c30acf9872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:25.095316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:25.108736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-280057c7-1257-4626-a022-85c30acf9872 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:25.110203 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-280057c7-1257-4626-a022-85c30acf9872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:25.189706 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 547/1081] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:25.209205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c2334-b9a8-43b3-9c86-bd5161c37cc9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:25.313135 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878c2334-b9a8-43b3-9c86-bd5161c37cc9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:25.343110 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:25.349553 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7842110-014e-4fd5-bf21-5d34b3c3e0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:25.350773 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7842110-014e-4fd5-bf21-5d34b3c3e0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:25.351405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7842110-014e-4fd5-bf21-5d34b3c3e0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:25.352018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7842110-014e-4fd5-bf21-5d34b3c3e0c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:25.420543 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7842110-014e-4fd5-bf21-5d34b3c3e0c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:25.422070 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7842110-014e-4fd5-bf21-5d34b3c3e0c9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:25.530384 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 535/1082] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:16:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 22:16:25.531955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280057c7-1257-4626-a022-85c30acf9872 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:25.637385 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280057c7-1257-4626-a022-85c30acf9872 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:25.666745 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:25.672570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f05bbf6-d1a6-447f-9771-6bb1c51831a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:25.673681 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05bbf6-d1a6-447f-9771-6bb1c51831a2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:25.674331 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05bbf6-d1a6-447f-9771-6bb1c51831a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:25.674971 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f05bbf6-d1a6-447f-9771-6bb1c51831a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:25.685516 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05bbf6-d1a6-447f-9771-6bb1c51831a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:25.686992 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f05bbf6-d1a6-447f-9771-6bb1c51831a2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:25.702148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-0f05bbf6-d1a6-447f-9771-6bb1c51831a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:25.702148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:25.702148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:25.702148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:25.702148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:25.702148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:25.702148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:25.702148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:25.702148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:25.702148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:25.702148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:25.702148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:25.702148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:16:25.702148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:16:25.702148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:16:25.703057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:16:25.703057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:25.703057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:25.703057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:25.703057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:25.703057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:25.703057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:25.703057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:25.703057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:25.703057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:25.703057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:25.703057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:25.703057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:25.703057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:25.703057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:25.703057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:25.703057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:25.709168 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:16:25.709168 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:16:25.709168 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:25.709168 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:25.709168 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 548/1083] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:25.710942 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7842110-014e-4fd5-bf21-5d34b3c3e0c9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:25.743491 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:25.818400 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7842110-014e-4fd5-bf21-5d34b3c3e0c9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:25.853951 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b3bac-ac78-4b90-8f93-9f0b4645c00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:25.854565 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b3bac-ac78-4b90-8f93-9f0b4645c00c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:25.854844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b3bac-ac78-4b90-8f93-9f0b4645c00c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:25.855119 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b3bac-ac78-4b90-8f93-9f0b4645c00c None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:25.855800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b3bac-ac78-4b90-8f93-9f0b4645c00c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:25.856104 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b3bac-ac78-4b90-8f93-9f0b4645c00c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:25.857460 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15b3bac-ac78-4b90-8f93-9f0b4645c00c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:25.877560 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 536/1084] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:26.371376 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d436a22-3280-42be-ac49-5d0118c0fa8e None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:26.371667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d436a22-3280-42be-ac49-5d0118c0fa8e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:26.371949 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d436a22-3280-42be-ac49-5d0118c0fa8e None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:26.373319 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 549/1085] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:26.378529 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25ce4a-3283-46b6-988f-59e2eb716e99 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:26.378965 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25ce4a-3283-46b6-988f-59e2eb716e99 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:26.379194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe25ce4a-3283-46b6-988f-59e2eb716e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:26.398767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-fe25ce4a-3283-46b6-988f-59e2eb716e99 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:26.399949 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fe25ce4a-3283-46b6-988f-59e2eb716e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:26.467230 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 537/1086] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:26.474617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216d52b-60cb-4df8-8484-87f9a30a46c2 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:26.475405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216d52b-60cb-4df8-8484-87f9a30a46c2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:26.475526 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216d52b-60cb-4df8-8484-87f9a30a46c2 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:26.476858 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 550/1087] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:26.481818 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c395e86-d7cc-48e2-ae28-cd0b6e41b366 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:26.482109 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c395e86-d7cc-48e2-ae28-cd0b6e41b366 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:26.482381 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c395e86-d7cc-48e2-ae28-cd0b6e41b366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:26.491140 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216d52b-60cb-4df8-8484-87f9a30a46c2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:26.511725 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5c395e86-d7cc-48e2-ae28-cd0b6e41b366 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:26.512915 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5c395e86-d7cc-48e2-ae28-cd0b6e41b366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:26.530098 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:26.583243 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 538/1088] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:16:26.593152 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216d52b-60cb-4df8-8484-87f9a30a46c2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:26.606874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c395e86-d7cc-48e2-ae28-cd0b6e41b366 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:26.629414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb27739-7376-481c-afdb-c3c5b4ebd0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:26.630073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb27739-7376-481c-afdb-c3c5b4ebd0ef None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:26.630362 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb27739-7376-481c-afdb-c3c5b4ebd0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:26.630638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb27739-7376-481c-afdb-c3c5b4ebd0ef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:26.641042 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb27739-7376-481c-afdb-c3c5b4ebd0ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:16:26.646077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-4fb27739-7376-481c-afdb-c3c5b4ebd0ef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 22:16:26.646077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:26.646077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:26.646077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:26.646077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:26.646077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:26.646077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:26.646077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:26.646077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:26.646077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:26.646077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:26.646077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:26.646077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:16:26.646077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:16:26.646077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:16:26.646077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:16:26.647321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:16:26.647321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:16:26.647321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:16:26.647321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:16:26.647321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:26.647321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:26.647321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:26.647321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:26.647321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:26.647321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:26.647321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:26.647321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:26.647321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:26.647321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:26.647321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:26.647321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:26.648291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:26.648291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:26.648291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:26.648291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:26.648291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:16:26.648291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:16:26.648291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:16:26.648291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:16:26.648291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:16:26.648291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:16:26.648291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 28 22:16:26.648291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:26.648291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 551/1089] 10.222.0.39 () {56 vars in 1120 bytes} [Thu Jul 28 22:16:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:26.653254 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb27739-7376-481c-afdb-c3c5b4ebd0ef None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:26.698049 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:26.708332 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c395e86-d7cc-48e2-ae28-cd0b6e41b366 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:26.745977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26a432-f654-4445-8ed6-e31c80016ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:26.746540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26a432-f654-4445-8ed6-e31c80016ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:26.746821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26a432-f654-4445-8ed6-e31c80016ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:26.747202 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd26a432-f654-4445-8ed6-e31c80016ab0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:26.757698 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd26a432-f654-4445-8ed6-e31c80016ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:16:26.760664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb27739-7376-481c-afdb-c3c5b4ebd0ef None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:26.762827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-fd26a432-f654-4445-8ed6-e31c80016ab0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:16:26.762827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:26.762827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:26.762827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:26.762827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:26.762827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:26.762827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:26.762827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:26.762827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:26.762827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:26.762827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:26.762827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:26.762827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:16:26.762827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:16:26.762827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:16:26.762827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:16:26.764214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:16:26.764214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:16:26.764214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:16:26.764214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:16:26.764214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:26.764214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:26.764214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:26.764214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:26.764214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:26.764214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:26.764214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:26.764214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:26.764214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:26.764214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:26.764214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:26.764214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:26.765134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:26.765134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:26.765134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:26.765134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:26.765134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:16:26.765134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:16:26.765134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:16:26.765134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:16:26.765134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:16:26.765134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:16:26.765134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:16:26.765134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:26.765134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 539/1090] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:16:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:26.770275 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26a432-f654-4445-8ed6-e31c80016ab0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:26.785651 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:26.802984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00be747-45f9-4d64-9095-9d5a5c682668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:26.804410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00be747-45f9-4d64-9095-9d5a5c682668 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:26.804699 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00be747-45f9-4d64-9095-9d5a5c682668 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:26.804980 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00be747-45f9-4d64-9095-9d5a5c682668 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:26.805727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00be747-45f9-4d64-9095-9d5a5c682668 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:26.806040 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00be747-45f9-4d64-9095-9d5a5c682668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:26.807424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00be747-45f9-4d64-9095-9d5a5c682668 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:26.815390 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 552/1091] 10.222.0.39 () {56 vars in 1115 bytes} [Thu Jul 28 22:16:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:26.821016 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00be747-45f9-4d64-9095-9d5a5c682668 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:26.874283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd26a432-f654-4445-8ed6-e31c80016ab0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:26.907832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3440d-d998-4e4a-a409-6a49a4005e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:26.908444 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3440d-d998-4e4a-a409-6a49a4005e92 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:26.908724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3440d-d998-4e4a-a409-6a49a4005e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:26.908997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3440d-d998-4e4a-a409-6a49a4005e92 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:26.909745 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3440d-d998-4e4a-a409-6a49a4005e92 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:26.910051 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3440d-d998-4e4a-a409-6a49a4005e92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:26.911414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3440d-d998-4e4a-a409-6a49a4005e92 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:26.920293 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 540/1092] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:16:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:26.921290 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00be747-45f9-4d64-9095-9d5a5c682668 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:26.925995 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3440d-d998-4e4a-a409-6a49a4005e92 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:26.929124 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:26.957050 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea932141-72fe-4991-b1f9-7dcb5c19a4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:26.957715 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea932141-72fe-4991-b1f9-7dcb5c19a4fb None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:26.957994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea932141-72fe-4991-b1f9-7dcb5c19a4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:26.958265 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea932141-72fe-4991-b1f9-7dcb5c19a4fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:26.969135 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea932141-72fe-4991-b1f9-7dcb5c19a4fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:26.970648 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea932141-72fe-4991-b1f9-7dcb5c19a4fb None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:26.985846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ea932141-72fe-4991-b1f9-7dcb5c19a4fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:26.985846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:26.985846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:26.985846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:26.985846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:26.985846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:26.985846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:26.985846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:26.985846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:26.985846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:26.985846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:26.985846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:26.985846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:16:26.985846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:16:26.985846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:16:26.986765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:16:26.986765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:26.986765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:26.986765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:26.986765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:26.986765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:26.986765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:26.986765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:26.986765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:26.986765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:26.986765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:26.986765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:26.986765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:26.986765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:26.986765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:26.986765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:26.986765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:26.987661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:16:26.987661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:16:26.987661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:26.987661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:26.987661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 553/1093] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:26.992774 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea932141-72fe-4991-b1f9-7dcb5c19a4fb None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:27.005373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:27.031745 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3440d-d998-4e4a-a409-6a49a4005e92 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:27.070053 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef522de-126b-4359-9af8-df15a3d62602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:27.070698 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef522de-126b-4359-9af8-df15a3d62602 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:27.070979 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef522de-126b-4359-9af8-df15a3d62602 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:27.071252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef522de-126b-4359-9af8-df15a3d62602 None admin] PATH_INFO: `/v3/users/55c6cdb859e147989561153f24a2dd78` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:27.072434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef522de-126b-4359-9af8-df15a3d62602 None admin] RBAC: Authorizing `identity:get_user(user_id=55c6cdb859e147989561153f24a2dd78)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:27.073942 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef522de-126b-4359-9af8-df15a3d62602 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:27.075558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 541/1094] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:16:26 2022] GET /identity/v3/users/55c6cdb859e147989561153f24a2dd78 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:27.081142 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef522de-126b-4359-9af8-df15a3d62602 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:27.103655 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea932141-72fe-4991-b1f9-7dcb5c19a4fb None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:27.139346 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3990a-6add-496c-9e09-f55e1807ca29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:27.139983 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3990a-6add-496c-9e09-f55e1807ca29 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:27.140269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3990a-6add-496c-9e09-f55e1807ca29 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:27.140550 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f3990a-6add-496c-9e09-f55e1807ca29 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:27.141245 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3990a-6add-496c-9e09-f55e1807ca29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:27.141591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3990a-6add-496c-9e09-f55e1807ca29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:27.142970 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f3990a-6add-496c-9e09-f55e1807ca29 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:27.158654 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 554/1095] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:27.165026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3990a-6add-496c-9e09-f55e1807ca29 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:27.183391 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:27.184657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef522de-126b-4359-9af8-df15a3d62602 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:27.218708 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ea0b3-8a78-4e63-8976-4525cd68aa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:27.219323 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ea0b3-8a78-4e63-8976-4525cd68aa69 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:27.219605 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ea0b3-8a78-4e63-8976-4525cd68aa69 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:27.219881 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ea0b3-8a78-4e63-8976-4525cd68aa69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:27.233099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:27.264042 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f3990a-6add-496c-9e09-f55e1807ca29 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:27.287929 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ea0b3-8a78-4e63-8976-4525cd68aa69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:27.289430 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ea0b3-8a78-4e63-8976-4525cd68aa69 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:27.298022 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f23b8-b135-4ac8-9bb7-0d1ae46dd36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:27.298666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f23b8-b135-4ac8-9bb7-0d1ae46dd36a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:27.298950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f23b8-b135-4ac8-9bb7-0d1ae46dd36a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:27.299229 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591f23b8-b135-4ac8-9bb7-0d1ae46dd36a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:27.307171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f23b8-b135-4ac8-9bb7-0d1ae46dd36a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:27.308904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591f23b8-b135-4ac8-9bb7-0d1ae46dd36a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:27.320661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-591f23b8-b135-4ac8-9bb7-0d1ae46dd36a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 28 22:16:27.320661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:27.320661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:27.320661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:27.320661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:27.320661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:27.320661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:27.320661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:27.320661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:27.320661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:27.320661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:27.320661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:27.320661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:16:27.320661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:16:27.320661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:16:27.320661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:16:27.321771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:27.321771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:27.321771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:16:27.321771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:16:27.321771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:16:27.321771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:16:27.321771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:27.321771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:27.321771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:27.321771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:27.321771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:27.321771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:27.321771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:27.321771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:27.321771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:27.321771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:27.321771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:27.321771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:27.323015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:27.323015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:27.323015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:16:27.323015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:16:27.323015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:16:27.323015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:16:27.323015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:16:27.323015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:16:27.323015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 28 22:16:27.323015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:27.323015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 555/1096] 10.222.0.39 () {56 vars in 1093 bytes} [Thu Jul 28 22:16:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:27.327446 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f23b8-b135-4ac8-9bb7-0d1ae46dd36a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:27.391612 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 542/1097] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:16:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 22:16:27.397609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ea0b3-8a78-4e63-8976-4525cd68aa69 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:27.407806 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:27.430765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591f23b8-b135-4ac8-9bb7-0d1ae46dd36a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:27.440834 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:27.464791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692a0f6-82f0-4f1f-b500-af69f540b44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:27.465403 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692a0f6-82f0-4f1f-b500-af69f540b44b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:27.465713 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692a0f6-82f0-4f1f-b500-af69f540b44b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:27.465989 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3692a0f6-82f0-4f1f-b500-af69f540b44b None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:27.466780 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692a0f6-82f0-4f1f-b500-af69f540b44b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:27.467092 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692a0f6-82f0-4f1f-b500-af69f540b44b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:27.468542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3692a0f6-82f0-4f1f-b500-af69f540b44b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:27.485904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 556/1098] 10.222.0.39 () {56 vars in 1133 bytes} [Thu Jul 28 22:16:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:27.492579 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692a0f6-82f0-4f1f-b500-af69f540b44b None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:27.497883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ea0b3-8a78-4e63-8976-4525cd68aa69 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:27.531461 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8bf028-6c17-4c67-b2b4-6ef7e9e5dad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:27.532031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bf028-6c17-4c67-b2b4-6ef7e9e5dad0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:27.532313 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bf028-6c17-4c67-b2b4-6ef7e9e5dad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:27.532585 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8bf028-6c17-4c67-b2b4-6ef7e9e5dad0 None admin] PATH_INFO: `/v3/projects/ccb474c36db842eea7c3658d041eb982` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:27.533765 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8bf028-6c17-4c67-b2b4-6ef7e9e5dad0 None admin] RBAC: Authorizing `identity:get_project(project_id=ccb474c36db842eea7c3658d041eb982)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:27.535193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8bf028-6c17-4c67-b2b4-6ef7e9e5dad0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:27.536857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 543/1099] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:16:27 2022] GET /identity/v3/projects/ccb474c36db842eea7c3658d041eb982 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:16:27.543610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8bf028-6c17-4c67-b2b4-6ef7e9e5dad0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:27.591077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3692a0f6-82f0-4f1f-b500-af69f540b44b None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:27.624936 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80778dd-2f1a-4262-94ea-48a7280387b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:27.625575 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80778dd-2f1a-4262-94ea-48a7280387b9 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:27.625838 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80778dd-2f1a-4262-94ea-48a7280387b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:27.626114 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80778dd-2f1a-4262-94ea-48a7280387b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:27.629396 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:27.642764 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8bf028-6c17-4c67-b2b4-6ef7e9e5dad0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:27.653338 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:27.677252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a38d5-98ad-4890-80b2-e2a135026c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:27.678565 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a38d5-98ad-4890-80b2-e2a135026c1d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:27.678791 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a38d5-98ad-4890-80b2-e2a135026c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:27.678992 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a38d5-98ad-4890-80b2-e2a135026c1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:27.679775 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a38d5-98ad-4890-80b2-e2a135026c1d None admin] RBAC: Adding query filter params (role.id=aded7bd43e974686b3c23bd9d4255b24, scope.project.id=ccb474c36db842eea7c3658d041eb982, user.id=55c6cdb859e147989561153f24a2dd78) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:27.680083 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a38d5-98ad-4890-80b2-e2a135026c1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:27.681501 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48a38d5-98ad-4890-80b2-e2a135026c1d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:27.687241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 544/1100] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:16:27 2022] GET /identity/v3/role_assignments?user.id=55c6cdb859e147989561153f24a2dd78&scope.project.id=ccb474c36db842eea7c3658d041eb982&role.id=aded7bd43e974686b3c23bd9d4255b24 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:27.696114 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80778dd-2f1a-4262-94ea-48a7280387b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:27.697747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80778dd-2f1a-4262-94ea-48a7280387b9 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:27.803929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 557/1101] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:16:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 28 22:16:27.809719 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a38d5-98ad-4890-80b2-e2a135026c1d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:27.912875 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a38d5-98ad-4890-80b2-e2a135026c1d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:27.946115 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52c34e-ce91-4d5c-abac-ebb0b5e40b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:27.946751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52c34e-ce91-4d5c-abac-ebb0b5e40b59 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:27.947032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52c34e-ce91-4d5c-abac-ebb0b5e40b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:27.947304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52c34e-ce91-4d5c-abac-ebb0b5e40b59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:27.957511 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52c34e-ce91-4d5c-abac-ebb0b5e40b59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:27.958966 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce52c34e-ce91-4d5c-abac-ebb0b5e40b59 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:27.972815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ce52c34e-ce91-4d5c-abac-ebb0b5e40b59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:27.972815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:27.972815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:27.972815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:27.972815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:27.972815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:27.972815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:27.972815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:27.972815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:27.972815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:27.972815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:27.972815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:27.972815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:16:27.972815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:16:27.972815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:16:27.973909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:16:27.973909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:27.973909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:27.973909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:27.973909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:27.973909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:27.973909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:27.973909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:27.973909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:27.973909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:27.973909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:27.973909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:27.973909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:27.973909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:27.973909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:27.973909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:27.973909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:27.974886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:16:27.974886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:16:27.974886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:16:27.974886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:27.977697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 545/1102] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:27.979743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80778dd-2f1a-4262-94ea-48a7280387b9 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:28.014442 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:28.079644 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80778dd-2f1a-4262-94ea-48a7280387b9 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:28.112966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386c0eb-b10e-4aae-95ae-9568cdf3dcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:28.113606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386c0eb-b10e-4aae-95ae-9568cdf3dcec None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:28.113886 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386c0eb-b10e-4aae-95ae-9568cdf3dcec None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:28.114156 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386c0eb-b10e-4aae-95ae-9568cdf3dcec None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:28.114841 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386c0eb-b10e-4aae-95ae-9568cdf3dcec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:28.115144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386c0eb-b10e-4aae-95ae-9568cdf3dcec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:28.116532 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8386c0eb-b10e-4aae-95ae-9568cdf3dcec None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:28.127684 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 558/1103] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:28.134520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52c34e-ce91-4d5c-abac-ebb0b5e40b59 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:28.166448 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:28.240609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce52c34e-ce91-4d5c-abac-ebb0b5e40b59 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:28.273982 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717962c3-7d22-436c-a007-2e792c5e568b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:28.274620 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717962c3-7d22-436c-a007-2e792c5e568b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:28.274891 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717962c3-7d22-436c-a007-2e792c5e568b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:28.275160 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717962c3-7d22-436c-a007-2e792c5e568b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:28.281573 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717962c3-7d22-436c-a007-2e792c5e568b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:28.283004 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717962c3-7d22-436c-a007-2e792c5e568b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:28.295001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-717962c3-7d22-436c-a007-2e792c5e568b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:16:28.295001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:28.295001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:28.295001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:28.295001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:28.295001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:28.295001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:28.295001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:28.295001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:28.295001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:28.295001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:28.295001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:28.295001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:16:28.295001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:16:28.295001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:16:28.295895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:16:28.295895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:28.295895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:28.295895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:28.295895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:28.295895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:28.295895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:28.295895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:28.295895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:28.295895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:28.295895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:28.295895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:28.295895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:28.295895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:28.295895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:28.295895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:28.295895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:28.296816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:16:28.296816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:16:28.296816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:16:28.296816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:16:28.296816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:16:28.296816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:16:28.296816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:16:28.296816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:28.297716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 546/1104] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:16:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:16:28.301860 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386c0eb-b10e-4aae-95ae-9568cdf3dcec None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:28.395019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:28.402197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386c0eb-b10e-4aae-95ae-9568cdf3dcec None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:28.434824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337dd74-548a-4966-b2df-1f4ce80c1cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:28.435441 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337dd74-548a-4966-b2df-1f4ce80c1cde None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:28.435827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337dd74-548a-4966-b2df-1f4ce80c1cde None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:28.436107 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337dd74-548a-4966-b2df-1f4ce80c1cde None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:28.436865 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337dd74-548a-4966-b2df-1f4ce80c1cde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:28.437177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337dd74-548a-4966-b2df-1f4ce80c1cde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:28.438662 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337dd74-548a-4966-b2df-1f4ce80c1cde None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:28.451936 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 559/1105] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:16:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:28.458019 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717962c3-7d22-436c-a007-2e792c5e568b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:28.541491 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:28.556612 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717962c3-7d22-436c-a007-2e792c5e568b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:28.588781 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d1d46a-404f-4c8d-aef5-6361012be5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:28.589378 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1d46a-404f-4c8d-aef5-6361012be5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:28.589682 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1d46a-404f-4c8d-aef5-6361012be5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:28.589954 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d1d46a-404f-4c8d-aef5-6361012be5d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:28.590728 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d1d46a-404f-4c8d-aef5-6361012be5d7 None admin] RBAC: Adding query filter params (role.id=62bb6d1764134942a1069bf3a5c1e0c4, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=77efe8046ea0468cb0c7c58c24fc4e75) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:28.591035 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d1d46a-404f-4c8d-aef5-6361012be5d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:28.592456 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d1d46a-404f-4c8d-aef5-6361012be5d7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:28.598092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 547/1106] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:16:28 2022] GET /identity/v3/role_assignments?user.id=77efe8046ea0468cb0c7c58c24fc4e75&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=62bb6d1764134942a1069bf3a5c1e0c4 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:29.744971 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c6848-add1-4ca1-a9f8-d5b352acb321 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:29.745502 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c6848-add1-4ca1-a9f8-d5b352acb321 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:29.745502 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c6848-add1-4ca1-a9f8-d5b352acb321 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:29.746898 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 560/1107] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:29.751876 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697a928-573c-4eab-ac35-efc5a3d91790 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:29.752291 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697a928-573c-4eab-ac35-efc5a3d91790 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:29.752420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697a928-573c-4eab-ac35-efc5a3d91790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:29.770827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5697a928-573c-4eab-ac35-efc5a3d91790 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:29.772715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5697a928-573c-4eab-ac35-efc5a3d91790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:29.839107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 548/1108] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:29.863680 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c6848-add1-4ca1-a9f8-d5b352acb321 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:29.961888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c6848-add1-4ca1-a9f8-d5b352acb321 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:29.994417 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baabce1-c6f0-49c3-938e-cfa3788eb81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:29.995088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baabce1-c6f0-49c3-938e-cfa3788eb81b None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:29.995347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baabce1-c6f0-49c3-938e-cfa3788eb81b None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:29.995620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baabce1-c6f0-49c3-938e-cfa3788eb81b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:29.996760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baabce1-c6f0-49c3-938e-cfa3788eb81b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:29.998216 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baabce1-c6f0-49c3-938e-cfa3788eb81b None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:29.999786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 561/1109] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:30.005374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697a928-573c-4eab-ac35-efc5a3d91790 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:30.082967 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:30.102890 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697a928-573c-4eab-ac35-efc5a3d91790 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:30.135172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1f2ae-71cf-486e-ab19-0ebf2b33d464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:30.135780 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1f2ae-71cf-486e-ab19-0ebf2b33d464 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:30.136060 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1f2ae-71cf-486e-ab19-0ebf2b33d464 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:30.136332 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1f2ae-71cf-486e-ab19-0ebf2b33d464 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:30.137006 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1f2ae-71cf-486e-ab19-0ebf2b33d464 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:30.138729 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1f2ae-71cf-486e-ab19-0ebf2b33d464 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:30.161068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 549/1110] 10.222.0.39 () {60 vars in 1132 bytes} [Thu Jul 28 22:16:30 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:16:32.177113 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8504245-fcd8-48c9-9e62-b5994c80b38f None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:32.177574 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8504245-fcd8-48c9-9e62-b5994c80b38f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:32.178302 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8504245-fcd8-48c9-9e62-b5994c80b38f None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:32.179610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 562/1111] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:32.184594 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407feba-254e-4f0f-96d1-35a2f861245b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:32.184963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407feba-254e-4f0f-96d1-35a2f861245b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:32.185144 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407feba-254e-4f0f-96d1-35a2f861245b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:32.202846 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7407feba-254e-4f0f-96d1-35a2f861245b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:32.204014 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7407feba-254e-4f0f-96d1-35a2f861245b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:32.268271 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 550/1112] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:32.291309 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8504245-fcd8-48c9-9e62-b5994c80b38f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:32.316387 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:32.391068 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8504245-fcd8-48c9-9e62-b5994c80b38f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:32.424272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f5367-8daa-4f7f-80d7-9a22c2db27eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:32.424839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f5367-8daa-4f7f-80d7-9a22c2db27eb None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:32.425128 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f5367-8daa-4f7f-80d7-9a22c2db27eb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:32.425396 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10f5367-8daa-4f7f-80d7-9a22c2db27eb None admin] PATH_INFO: `/v3/roles/4146966c23674402aebb9a5b21cf18ba` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:32.426864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f5367-8daa-4f7f-80d7-9a22c2db27eb None admin] RBAC: Authorizing `identity:get_role(role_id=4146966c23674402aebb9a5b21cf18ba)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:32.428249 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10f5367-8daa-4f7f-80d7-9a22c2db27eb None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:32.429593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 563/1113] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:16:32 2022] GET /identity/v3/roles/4146966c23674402aebb9a5b21cf18ba => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:32.435331 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407feba-254e-4f0f-96d1-35a2f861245b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:32.504663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:32.539635 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407feba-254e-4f0f-96d1-35a2f861245b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:32.573267 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58b055-8227-4635-a883-f50b91bb718e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:32.573929 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58b055-8227-4635-a883-f50b91bb718e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:32.574213 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58b055-8227-4635-a883-f50b91bb718e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:32.574489 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c58b055-8227-4635-a883-f50b91bb718e None admin] PATH_INFO: `/v3/users/58599ae6271a469fb7443ac1e03fcf89` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:32.587142 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58b055-8227-4635-a883-f50b91bb718e None admin] RBAC: Authorizing `identity:get_user(user_id=58599ae6271a469fb7443ac1e03fcf89)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:32.588587 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c58b055-8227-4635-a883-f50b91bb718e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:32.590231 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 551/1114] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:16:32 2022] GET /identity/v3/users/58599ae6271a469fb7443ac1e03fcf89 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:32.596076 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f5367-8daa-4f7f-80d7-9a22c2db27eb None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:32.686059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:32.695326 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10f5367-8daa-4f7f-80d7-9a22c2db27eb None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:32.727934 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6105b018-a3d5-4850-80aa-05a695ac1e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:32.728541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105b018-a3d5-4850-80aa-05a695ac1e09 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:32.728826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105b018-a3d5-4850-80aa-05a695ac1e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:32.729104 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6105b018-a3d5-4850-80aa-05a695ac1e09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:32.797743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6105b018-a3d5-4850-80aa-05a695ac1e09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:32.799216 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6105b018-a3d5-4850-80aa-05a695ac1e09 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:32.888657 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:32.898134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 564/1115] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:16:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 22:16:32.904136 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58b055-8227-4635-a883-f50b91bb718e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:33.005206 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c58b055-8227-4635-a883-f50b91bb718e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:33.029127 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:33.040471 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcbe40-4cbd-4b04-8a6a-4099af06cbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:33.041107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcbe40-4cbd-4b04-8a6a-4099af06cbb9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:33.041388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcbe40-4cbd-4b04-8a6a-4099af06cbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:33.041691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcbe40-4cbd-4b04-8a6a-4099af06cbb9 None admin] PATH_INFO: `/v3/projects/ccb474c36db842eea7c3658d041eb982` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:33.042835 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcbe40-4cbd-4b04-8a6a-4099af06cbb9 None admin] RBAC: Authorizing `identity:get_project(project_id=ccb474c36db842eea7c3658d041eb982)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:33.044266 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcbe40-4cbd-4b04-8a6a-4099af06cbb9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:33.045928 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 552/1116] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:16:32 2022] GET /identity/v3/projects/ccb474c36db842eea7c3658d041eb982 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:33.052026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6105b018-a3d5-4850-80aa-05a695ac1e09 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:33.147924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6105b018-a3d5-4850-80aa-05a695ac1e09 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:33.181484 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f8cae-8c25-4492-be3e-41f2715ce2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:33.182121 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f8cae-8c25-4492-be3e-41f2715ce2be None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:33.182398 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f8cae-8c25-4492-be3e-41f2715ce2be None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:33.182664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f8cae-8c25-4492-be3e-41f2715ce2be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:33.183439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f8cae-8c25-4492-be3e-41f2715ce2be None admin] RBAC: Adding query filter params (role.id=4146966c23674402aebb9a5b21cf18ba, scope.project.id=ccb474c36db842eea7c3658d041eb982, user.id=58599ae6271a469fb7443ac1e03fcf89) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:33.183745 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f8cae-8c25-4492-be3e-41f2715ce2be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:33.185150 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f8cae-8c25-4492-be3e-41f2715ce2be None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:33.190462 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 565/1117] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:16:33 2022] GET /identity/v3/role_assignments?user.id=58599ae6271a469fb7443ac1e03fcf89&scope.project.id=ccb474c36db842eea7c3658d041eb982&role.id=4146966c23674402aebb9a5b21cf18ba => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:35.144482 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c142046-26d9-4353-a11d-1aee2cfdb3a9 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:35.145343 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c142046-26d9-4353-a11d-1aee2cfdb3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:35.145343 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c142046-26d9-4353-a11d-1aee2cfdb3a9 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:35.146976 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 553/1118] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:35.152440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681afbf-0446-4ba7-b0bc-3b810fa2d4ca None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:35.152881 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681afbf-0446-4ba7-b0bc-3b810fa2d4ca None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:35.153028 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681afbf-0446-4ba7-b0bc-3b810fa2d4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:35.171455 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c681afbf-0446-4ba7-b0bc-3b810fa2d4ca None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:35.172655 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c681afbf-0446-4ba7-b0bc-3b810fa2d4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:35.221711 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:35.238271 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 566/1119] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:35.262090 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c142046-26d9-4353-a11d-1aee2cfdb3a9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:35.360783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c142046-26d9-4353-a11d-1aee2cfdb3a9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:35.392696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4dbe7-aa8a-4321-a44e-2c8dc768a775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:35.393314 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4dbe7-aa8a-4321-a44e-2c8dc768a775 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:35.393614 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4dbe7-aa8a-4321-a44e-2c8dc768a775 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:35.393886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4dbe7-aa8a-4321-a44e-2c8dc768a775 None admin] PATH_INFO: `/v3/roles/4146966c23674402aebb9a5b21cf18ba` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:35.395347 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4dbe7-aa8a-4321-a44e-2c8dc768a775 None admin] RBAC: Authorizing `identity:get_role(role_id=4146966c23674402aebb9a5b21cf18ba)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:35.397319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c4dbe7-aa8a-4321-a44e-2c8dc768a775 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:35.398651 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 554/1120] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:16:35 2022] GET /identity/v3/roles/4146966c23674402aebb9a5b21cf18ba => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:35.404464 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681afbf-0446-4ba7-b0bc-3b810fa2d4ca None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:35.503992 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681afbf-0446-4ba7-b0bc-3b810fa2d4ca None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:35.536711 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a748737-f3e1-4933-ab97-56761ff4dc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:35.538100 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a748737-f3e1-4933-ab97-56761ff4dc42 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:35.538381 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a748737-f3e1-4933-ab97-56761ff4dc42 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:35.538654 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a748737-f3e1-4933-ab97-56761ff4dc42 None admin] PATH_INFO: `/v3/users/58599ae6271a469fb7443ac1e03fcf89` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:35.539803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a748737-f3e1-4933-ab97-56761ff4dc42 None admin] RBAC: Authorizing `identity:get_user(user_id=58599ae6271a469fb7443ac1e03fcf89)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:35.541212 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a748737-f3e1-4933-ab97-56761ff4dc42 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:35.542822 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 567/1121] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:16:35 2022] GET /identity/v3/users/58599ae6271a469fb7443ac1e03fcf89 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:35.548530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4dbe7-aa8a-4321-a44e-2c8dc768a775 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:35.607417 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:35.648759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4dbe7-aa8a-4321-a44e-2c8dc768a775 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:35.681357 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5619dda-9f41-4728-a15b-24a5317a3fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:35.682013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5619dda-9f41-4728-a15b-24a5317a3fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:35.682295 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5619dda-9f41-4728-a15b-24a5317a3fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:35.682568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5619dda-9f41-4728-a15b-24a5317a3fb7 None admin] PATH_INFO: `/v3/projects/ccb474c36db842eea7c3658d041eb982` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:35.683707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5619dda-9f41-4728-a15b-24a5317a3fb7 None admin] RBAC: Authorizing `identity:get_project(project_id=ccb474c36db842eea7c3658d041eb982)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:35.685130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5619dda-9f41-4728-a15b-24a5317a3fb7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:35.686785 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 555/1122] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:16:35 2022] GET /identity/v3/projects/ccb474c36db842eea7c3658d041eb982 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:35.692664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a748737-f3e1-4933-ab97-56761ff4dc42 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:35.746857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:35.791278 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a748737-f3e1-4933-ab97-56761ff4dc42 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:35.823846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f699cf-2f4f-40ba-92df-aa948c9bd0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:35.824507 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f699cf-2f4f-40ba-92df-aa948c9bd0a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:35.824791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f699cf-2f4f-40ba-92df-aa948c9bd0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:35.825069 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f699cf-2f4f-40ba-92df-aa948c9bd0a1 None admin] PATH_INFO: `/v3/projects/ccb474c36db842eea7c3658d041eb982/users/58599ae6271a469fb7443ac1e03fcf89/roles/4146966c23674402aebb9a5b21cf18ba` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:35.827241 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f699cf-2f4f-40ba-92df-aa948c9bd0a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=ccb474c36db842eea7c3658d041eb982, user_id=58599ae6271a469fb7443ac1e03fcf89, role_id=4146966c23674402aebb9a5b21cf18ba)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:35.828672 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f699cf-2f4f-40ba-92df-aa948c9bd0a1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:35.842297 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 568/1123] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:16:35 2022] PUT /identity/v3/projects/ccb474c36db842eea7c3658d041eb982/users/58599ae6271a469fb7443ac1e03fcf89/roles/4146966c23674402aebb9a5b21cf18ba => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:16:37.816639 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afde79-6100-42f2-b94c-1bdfe8969209 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:37.817173 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afde79-6100-42f2-b94c-1bdfe8969209 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:37.817173 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afde79-6100-42f2-b94c-1bdfe8969209 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:37.819115 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 556/1124] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:37.824283 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7b4ed-82ee-4005-8ded-b7774b3e9a7f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:37.824715 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7b4ed-82ee-4005-8ded-b7774b3e9a7f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:37.824845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b7b4ed-82ee-4005-8ded-b7774b3e9a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:37.843287 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e1b7b4ed-82ee-4005-8ded-b7774b3e9a7f None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:37.844474 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e1b7b4ed-82ee-4005-8ded-b7774b3e9a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:37.909387 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 569/1125] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:37.932008 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afde79-6100-42f2-b94c-1bdfe8969209 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:38.031985 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afde79-6100-42f2-b94c-1bdfe8969209 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:38.057205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:38.064782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea275e9-21c3-432a-9e92-a376f4ff9ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:38.065409 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea275e9-21c3-432a-9e92-a376f4ff9ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:38.065717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea275e9-21c3-432a-9e92-a376f4ff9ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:38.065995 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea275e9-21c3-432a-9e92-a376f4ff9ed1 None admin] PATH_INFO: `/v3/roles/4146966c23674402aebb9a5b21cf18ba` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:38.067477 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea275e9-21c3-432a-9e92-a376f4ff9ed1 None admin] RBAC: Authorizing `identity:get_role(role_id=4146966c23674402aebb9a5b21cf18ba)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:38.069307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea275e9-21c3-432a-9e92-a376f4ff9ed1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:38.070658 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 557/1126] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:16:37 2022] GET /identity/v3/roles/4146966c23674402aebb9a5b21cf18ba => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:38.076349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7b4ed-82ee-4005-8ded-b7774b3e9a7f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:38.119319 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:38.172729 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b7b4ed-82ee-4005-8ded-b7774b3e9a7f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:38.205209 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fa2ca-d942-4994-aca4-077b45c16fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:38.205862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fa2ca-d942-4994-aca4-077b45c16fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:38.206141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fa2ca-d942-4994-aca4-077b45c16fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:38.206409 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fa2ca-d942-4994-aca4-077b45c16fa4 None admin] PATH_INFO: `/v3/users/58599ae6271a469fb7443ac1e03fcf89` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:38.207559 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fa2ca-d942-4994-aca4-077b45c16fa4 None admin] RBAC: Authorizing `identity:get_user(user_id=58599ae6271a469fb7443ac1e03fcf89)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:38.209587 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8fa2ca-d942-4994-aca4-077b45c16fa4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:38.211232 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 570/1127] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:16:38 2022] GET /identity/v3/users/58599ae6271a469fb7443ac1e03fcf89 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:38.216888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea275e9-21c3-432a-9e92-a376f4ff9ed1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:38.312897 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea275e9-21c3-432a-9e92-a376f4ff9ed1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:38.345141 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a531f7-e6c1-46da-907b-202d1766b51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:38.345768 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a531f7-e6c1-46da-907b-202d1766b51b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:38.346047 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a531f7-e6c1-46da-907b-202d1766b51b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:38.346316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a531f7-e6c1-46da-907b-202d1766b51b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:38.404582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:38.411337 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a531f7-e6c1-46da-907b-202d1766b51b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:38.412795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a531f7-e6c1-46da-907b-202d1766b51b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:38.507390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 558/1128] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:16:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 22:16:38.514234 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fa2ca-d942-4994-aca4-077b45c16fa4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:38.612957 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8fa2ca-d942-4994-aca4-077b45c16fa4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:38.624989 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:38.647257 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af861178-8227-4382-9529-dd4272afb9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:38.647907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af861178-8227-4382-9529-dd4272afb9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:38.648187 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af861178-8227-4382-9529-dd4272afb9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:38.648457 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af861178-8227-4382-9529-dd4272afb9d8 None admin] PATH_INFO: `/v3/projects/ccb474c36db842eea7c3658d041eb982` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:38.649653 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af861178-8227-4382-9529-dd4272afb9d8 None admin] RBAC: Authorizing `identity:get_project(project_id=ccb474c36db842eea7c3658d041eb982)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:38.651076 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af861178-8227-4382-9529-dd4272afb9d8 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:38.652729 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 571/1129] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:16:38 2022] GET /identity/v3/projects/ccb474c36db842eea7c3658d041eb982 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:38.658944 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a531f7-e6c1-46da-907b-202d1766b51b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:38.748001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:38.753886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a531f7-e6c1-46da-907b-202d1766b51b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:38.797637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4eab9e-9d47-406f-a505-a61a88a1a956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:38.798258 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4eab9e-9d47-406f-a505-a61a88a1a956 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:38.798538 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4eab9e-9d47-406f-a505-a61a88a1a956 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:38.798810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4eab9e-9d47-406f-a505-a61a88a1a956 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:38.799634 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4eab9e-9d47-406f-a505-a61a88a1a956 None admin] RBAC: Adding query filter params (role.id=4146966c23674402aebb9a5b21cf18ba, scope.project.id=ccb474c36db842eea7c3658d041eb982, user.id=58599ae6271a469fb7443ac1e03fcf89) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:38.799945 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4eab9e-9d47-406f-a505-a61a88a1a956 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:38.801369 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4eab9e-9d47-406f-a505-a61a88a1a956 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:38.807289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 559/1130] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:16:38 2022] GET /identity/v3/role_assignments?user.id=58599ae6271a469fb7443ac1e03fcf89&scope.project.id=ccb474c36db842eea7c3658d041eb982&role.id=4146966c23674402aebb9a5b21cf18ba => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:40.834716 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b0f6c-c220-4d24-a33f-2e0c28ce0a16 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:40.835268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b0f6c-c220-4d24-a33f-2e0c28ce0a16 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:40.835268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2b0f6c-c220-4d24-a33f-2e0c28ce0a16 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:40.836619 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 572/1131] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:40.841720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3fa58-30e7-4dae-8ad1-2159d50e0a57 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:40.842106 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3fa58-30e7-4dae-8ad1-2159d50e0a57 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:40.842275 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3fa58-30e7-4dae-8ad1-2159d50e0a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:40.860537 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-39c3fa58-30e7-4dae-8ad1-2159d50e0a57 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:40.861749 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-39c3fa58-30e7-4dae-8ad1-2159d50e0a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:40.927044 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 560/1132] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:40.950742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b0f6c-c220-4d24-a33f-2e0c28ce0a16 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:41.050159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2b0f6c-c220-4d24-a33f-2e0c28ce0a16 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:41.082630 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b64a61-8910-4495-85a0-3da3f3dbcd59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:41.083254 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b64a61-8910-4495-85a0-3da3f3dbcd59 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:41.083583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b64a61-8910-4495-85a0-3da3f3dbcd59 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:41.083971 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b64a61-8910-4495-85a0-3da3f3dbcd59 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:41.085162 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b64a61-8910-4495-85a0-3da3f3dbcd59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:41.086558 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b64a61-8910-4495-85a0-3da3f3dbcd59 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:41.088132 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 573/1133] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:41.093899 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3fa58-30e7-4dae-8ad1-2159d50e0a57 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:41.186512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:41.196991 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3fa58-30e7-4dae-8ad1-2159d50e0a57 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:41.231344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486661d3-f115-4e72-bbd3-2877e2ce2c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:41.231948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486661d3-f115-4e72-bbd3-2877e2ce2c50 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:41.232232 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486661d3-f115-4e72-bbd3-2877e2ce2c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:41.232511 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486661d3-f115-4e72-bbd3-2877e2ce2c50 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:41.233183 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486661d3-f115-4e72-bbd3-2877e2ce2c50 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:41.234647 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486661d3-f115-4e72-bbd3-2877e2ce2c50 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:41.262101 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 561/1134] 10.222.0.39 () {60 vars in 1141 bytes} [Thu Jul 28 22:16:41 2022] POST /identity/v3/projects => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:16:43.287518 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c65cd-d908-4f01-9ef0-3a12dd6df40d None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:43.288068 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c65cd-d908-4f01-9ef0-3a12dd6df40d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:43.288068 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c65cd-d908-4f01-9ef0-3a12dd6df40d None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:43.289422 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 574/1135] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:43.294753 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c17808-0e22-4bbc-87c2-a824f0d286f6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:43.295133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c17808-0e22-4bbc-87c2-a824f0d286f6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:43.295318 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c17808-0e22-4bbc-87c2-a824f0d286f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:43.312973 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-07c17808-0e22-4bbc-87c2-a824f0d286f6 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:43.314203 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-07c17808-0e22-4bbc-87c2-a824f0d286f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:43.380833 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 562/1136] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:43.405394 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c65cd-d908-4f01-9ef0-3a12dd6df40d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:43.453873 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:43.504773 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c65cd-d908-4f01-9ef0-3a12dd6df40d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:43.538410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9088fa81-8fff-4a94-9483-057dc81933c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:43.539024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088fa81-8fff-4a94-9483-057dc81933c6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:43.539303 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088fa81-8fff-4a94-9483-057dc81933c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:43.539573 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088fa81-8fff-4a94-9483-057dc81933c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:43.540708 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9088fa81-8fff-4a94-9483-057dc81933c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:43.542155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9088fa81-8fff-4a94-9483-057dc81933c6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:43.543720 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 575/1137] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:16:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:43.549315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c17808-0e22-4bbc-87c2-a824f0d286f6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:43.618233 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:43.648328 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c17808-0e22-4bbc-87c2-a824f0d286f6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:43.680994 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efafc7ea-c986-4c0b-b84f-d6e963258a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:43.681558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafc7ea-c986-4c0b-b84f-d6e963258a04 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:43.681841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafc7ea-c986-4c0b-b84f-d6e963258a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:43.682111 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efafc7ea-c986-4c0b-b84f-d6e963258a04 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:43.682792 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efafc7ea-c986-4c0b-b84f-d6e963258a04 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:43.684210 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efafc7ea-c986-4c0b-b84f-d6e963258a04 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:43.705100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 563/1138] 10.222.0.39 () {60 vars in 1132 bytes} [Thu Jul 28 22:16:43 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:16:45.722660 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5149f9-7cfc-4fd0-83b7-8d871cd59bb1 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:45.723190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5149f9-7cfc-4fd0-83b7-8d871cd59bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:45.723190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5149f9-7cfc-4fd0-83b7-8d871cd59bb1 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:45.724716 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 576/1139] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:45.729831 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04547f37-64a2-42b9-ad2d-9b2b9f0fc8ac None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:45.730245 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04547f37-64a2-42b9-ad2d-9b2b9f0fc8ac None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:45.730368 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04547f37-64a2-42b9-ad2d-9b2b9f0fc8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:45.748458 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-04547f37-64a2-42b9-ad2d-9b2b9f0fc8ac None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:45.749735 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-04547f37-64a2-42b9-ad2d-9b2b9f0fc8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:45.815703 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 564/1140] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:16:45.838823 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5149f9-7cfc-4fd0-83b7-8d871cd59bb1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:45.935972 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5149f9-7cfc-4fd0-83b7-8d871cd59bb1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:45.952094 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:45.969500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bbd156-c94c-437c-b1dc-f59415710684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:45.970150 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbd156-c94c-437c-b1dc-f59415710684 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:45.970428 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbd156-c94c-437c-b1dc-f59415710684 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:45.970705 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bbd156-c94c-437c-b1dc-f59415710684 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:45.980462 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bbd156-c94c-437c-b1dc-f59415710684 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:16:45.985280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-54bbd156-c94c-437c-b1dc-f59415710684 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:16:45.985280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:45.985280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:45.985280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:45.985280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:45.985280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:45.985280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:45.985280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:45.985280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:45.985280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:45.985280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:45.985280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:45.985280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:16:45.985280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:16:45.985280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:16:45.985280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:16:45.986491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:16:45.986491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:16:45.986491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:16:45.986491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:16:45.986491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:45.986491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:45.986491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:45.986491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:45.986491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:45.986491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:45.986491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:45.986491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:45.986491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:45.986491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:45.986491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:45.986491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:45.987636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:45.987636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:45.987636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:45.987636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:45.987636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:16:45.987636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:16:45.987636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:16:45.987636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:16:45.987636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:16:45.987636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:16:45.987636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:16:45.987636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:45.987636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 577/1141] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:16:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:45.992998 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04547f37-64a2-42b9-ad2d-9b2b9f0fc8ac None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:46.050336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:46.093520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04547f37-64a2-42b9-ad2d-9b2b9f0fc8ac None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:46.127931 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7824b-47d4-44ae-a05f-e92701da22f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:46.128534 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7824b-47d4-44ae-a05f-e92701da22f2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:46.128814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7824b-47d4-44ae-a05f-e92701da22f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:46.129087 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7824b-47d4-44ae-a05f-e92701da22f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:46.129829 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7824b-47d4-44ae-a05f-e92701da22f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:46.130403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7824b-47d4-44ae-a05f-e92701da22f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:46.131852 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7824b-47d4-44ae-a05f-e92701da22f2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:46.140003 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 565/1142] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:16:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:46.145786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bbd156-c94c-437c-b1dc-f59415710684 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:46.243811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bbd156-c94c-437c-b1dc-f59415710684 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:46.276368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb2970-622a-4980-863d-90ec3296b301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:46.276982 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb2970-622a-4980-863d-90ec3296b301 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:46.277265 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb2970-622a-4980-863d-90ec3296b301 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:46.277560 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fb2970-622a-4980-863d-90ec3296b301 None admin] PATH_INFO: `/v3/users/3b562902ef404a5dbdb3287d4f82bfdb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:46.290745 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb2970-622a-4980-863d-90ec3296b301 None admin] RBAC: Authorizing `identity:get_user(user_id=3b562902ef404a5dbdb3287d4f82bfdb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:46.292220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fb2970-622a-4980-863d-90ec3296b301 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:46.293871 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 578/1143] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:16:46 2022] GET /identity/v3/users/3b562902ef404a5dbdb3287d4f82bfdb => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:46.299593 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7824b-47d4-44ae-a05f-e92701da22f2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:46.399214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7824b-47d4-44ae-a05f-e92701da22f2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:46.418461 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:46.432153 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a36f4-8c5e-4f42-9b6f-95bbbda1bb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:46.432762 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a36f4-8c5e-4f42-9b6f-95bbbda1bb91 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:46.433042 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a36f4-8c5e-4f42-9b6f-95bbbda1bb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:46.433312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a36f4-8c5e-4f42-9b6f-95bbbda1bb91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:46.498422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a36f4-8c5e-4f42-9b6f-95bbbda1bb91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:46.499771 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a36f4-8c5e-4f42-9b6f-95bbbda1bb91 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:46.598438 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 566/1144] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:16:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 22:16:46.604363 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb2970-622a-4980-863d-90ec3296b301 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:46.638558 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:46.702534 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fb2970-622a-4980-863d-90ec3296b301 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:46.734841 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1428bb-8f89-4acb-a5f0-92672ec3f068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:46.735460 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1428bb-8f89-4acb-a5f0-92672ec3f068 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:46.735740 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1428bb-8f89-4acb-a5f0-92672ec3f068 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:46.736016 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1428bb-8f89-4acb-a5f0-92672ec3f068 None admin] PATH_INFO: `/v3/projects/a2c261edaf464ce0872c694ae369019d` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:46.737147 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1428bb-8f89-4acb-a5f0-92672ec3f068 None admin] RBAC: Authorizing `identity:get_project(project_id=a2c261edaf464ce0872c694ae369019d)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:46.738585 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1428bb-8f89-4acb-a5f0-92672ec3f068 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:46.740843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 579/1145] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:16:46 2022] GET /identity/v3/projects/a2c261edaf464ce0872c694ae369019d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:46.746801 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a36f4-8c5e-4f42-9b6f-95bbbda1bb91 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:46.768761 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:46.846789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a36f4-8c5e-4f42-9b6f-95bbbda1bb91 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:46.881950 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002afbeb-e6fa-4dd7-916b-3e419a36a04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:46.882566 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002afbeb-e6fa-4dd7-916b-3e419a36a04d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:46.882846 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002afbeb-e6fa-4dd7-916b-3e419a36a04d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:46.883119 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002afbeb-e6fa-4dd7-916b-3e419a36a04d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:46.883905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002afbeb-e6fa-4dd7-916b-3e419a36a04d None admin] RBAC: Adding query filter params (role.id=aded7bd43e974686b3c23bd9d4255b24, scope.project.id=a2c261edaf464ce0872c694ae369019d, user.id=3b562902ef404a5dbdb3287d4f82bfdb) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:46.884215 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002afbeb-e6fa-4dd7-916b-3e419a36a04d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:46.885664 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002afbeb-e6fa-4dd7-916b-3e419a36a04d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:46.891186 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 567/1146] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:16:46 2022] GET /identity/v3/role_assignments?user.id=3b562902ef404a5dbdb3287d4f82bfdb&scope.project.id=a2c261edaf464ce0872c694ae369019d&role.id=aded7bd43e974686b3c23bd9d4255b24 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:48.849998 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485204a-2a00-48ed-a2ef-908cbdb15ab0 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:48.850555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485204a-2a00-48ed-a2ef-908cbdb15ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:48.850555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485204a-2a00-48ed-a2ef-908cbdb15ab0 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:48.851929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 580/1147] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:48.857165 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e908fc-5215-4362-9e5a-658b348f099d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:48.857638 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e908fc-5215-4362-9e5a-658b348f099d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:48.857769 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e908fc-5215-4362-9e5a-658b348f099d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:48.876287 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-22e908fc-5215-4362-9e5a-658b348f099d None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:48.877495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-22e908fc-5215-4362-9e5a-658b348f099d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:48.944484 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 568/1148] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:48.966372 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485204a-2a00-48ed-a2ef-908cbdb15ab0 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:49.064112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485204a-2a00-48ed-a2ef-908cbdb15ab0 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:49.065595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:49.098330 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbbd2d-0ee0-4826-9c54-cb17e56ff0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:49.099604 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbbd2d-0ee0-4826-9c54-cb17e56ff0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:49.099890 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbbd2d-0ee0-4826-9c54-cb17e56ff0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:49.100220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cbbd2d-0ee0-4826-9c54-cb17e56ff0f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:49.109857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cbbd2d-0ee0-4826-9c54-cb17e56ff0f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:16:49.115666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-37cbbd2d-0ee0-4826-9c54-cb17e56ff0f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:16:49.115666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:49.115666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:49.115666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:49.115666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:49.115666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:49.115666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:49.115666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:49.115666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:49.115666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:49.115666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:49.115666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:49.115666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:16:49.115666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:16:49.115666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:16:49.115666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:16:49.116857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:16:49.116857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:16:49.116857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:16:49.116857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:16:49.116857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:49.116857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:49.116857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:49.116857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:49.116857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:49.116857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:49.116857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:49.116857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:49.116857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:49.116857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:49.116857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:49.116857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:49.117759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:49.117759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:49.117759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:49.117759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:49.117759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:16:49.117759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:16:49.117759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:16:49.117759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:16:49.117759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:16:49.117759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:16:49.117759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:16:49.117759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:49.117759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 581/1149] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:16:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:49.122795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e908fc-5215-4362-9e5a-658b348f099d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:49.137413 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:49.224130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e908fc-5215-4362-9e5a-658b348f099d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:49.257350 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f7603-837f-4417-aae4-15d3aeb616e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:49.257988 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f7603-837f-4417-aae4-15d3aeb616e5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:49.258270 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f7603-837f-4417-aae4-15d3aeb616e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:49.258547 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6f7603-837f-4417-aae4-15d3aeb616e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:49.259259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f7603-837f-4417-aae4-15d3aeb616e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:49.259564 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f7603-837f-4417-aae4-15d3aeb616e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:49.260933 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6f7603-837f-4417-aae4-15d3aeb616e5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:49.268970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 569/1150] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:16:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:49.275509 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbbd2d-0ee0-4826-9c54-cb17e56ff0f5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:49.376057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cbbd2d-0ee0-4826-9c54-cb17e56ff0f5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:49.410153 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76615244-a173-4499-b3aa-7eec25ceb10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:49.410771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76615244-a173-4499-b3aa-7eec25ceb10d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:49.411052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76615244-a173-4499-b3aa-7eec25ceb10d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:49.412357 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76615244-a173-4499-b3aa-7eec25ceb10d None admin] PATH_INFO: `/v3/users/3b562902ef404a5dbdb3287d4f82bfdb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:49.413550 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76615244-a173-4499-b3aa-7eec25ceb10d None admin] RBAC: Authorizing `identity:get_user(user_id=3b562902ef404a5dbdb3287d4f82bfdb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:49.414976 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76615244-a173-4499-b3aa-7eec25ceb10d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:49.416556 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 582/1151] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:16:49 2022] GET /identity/v3/users/3b562902ef404a5dbdb3287d4f82bfdb => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:49.422314 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f7603-837f-4417-aae4-15d3aeb616e5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:49.501543 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:49.524984 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6f7603-837f-4417-aae4-15d3aeb616e5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:49.557979 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f245df94-9528-4384-a64c-4ab6022efe03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:49.558613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245df94-9528-4384-a64c-4ab6022efe03 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:49.558898 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245df94-9528-4384-a64c-4ab6022efe03 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:49.559175 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f245df94-9528-4384-a64c-4ab6022efe03 None admin] PATH_INFO: `/v3/projects/a2c261edaf464ce0872c694ae369019d` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:49.560325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f245df94-9528-4384-a64c-4ab6022efe03 None admin] RBAC: Authorizing `identity:get_project(project_id=a2c261edaf464ce0872c694ae369019d)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:49.561792 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f245df94-9528-4384-a64c-4ab6022efe03 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:49.563442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 570/1152] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:16:49 2022] GET /identity/v3/projects/a2c261edaf464ce0872c694ae369019d => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:49.569715 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76615244-a173-4499-b3aa-7eec25ceb10d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:49.600406 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:49.667492 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76615244-a173-4499-b3aa-7eec25ceb10d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:49.698856 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2152a616-d1da-47a3-bee5-4519920977fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:49.699499 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152a616-d1da-47a3-bee5-4519920977fb None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:49.699778 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152a616-d1da-47a3-bee5-4519920977fb None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:49.700050 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2152a616-d1da-47a3-bee5-4519920977fb None admin] PATH_INFO: `/v3/projects/a2c261edaf464ce0872c694ae369019d/users/3b562902ef404a5dbdb3287d4f82bfdb/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:49.701959 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2152a616-d1da-47a3-bee5-4519920977fb None admin] RBAC: Authorizing `identity:create_grant(project_id=a2c261edaf464ce0872c694ae369019d, user_id=3b562902ef404a5dbdb3287d4f82bfdb, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:49.703372 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2152a616-d1da-47a3-bee5-4519920977fb None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:49.716473 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 583/1153] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:16:49 2022] PUT /identity/v3/projects/a2c261edaf464ce0872c694ae369019d/users/3b562902ef404a5dbdb3287d4f82bfdb/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:16:51.689161 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cb06c-18d8-47f0-bfb3-e448136acff6 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:51.689729 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cb06c-18d8-47f0-bfb3-e448136acff6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:51.689729 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cb06c-18d8-47f0-bfb3-e448136acff6 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:51.691062 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 571/1154] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:51.696439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8f505-c8c3-4bf1-b6ca-26e5daac3461 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:51.696878 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8f505-c8c3-4bf1-b6ca-26e5daac3461 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:51.697003 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db8f505-c8c3-4bf1-b6ca-26e5daac3461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:51.715561 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1db8f505-c8c3-4bf1-b6ca-26e5daac3461 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:51.716746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1db8f505-c8c3-4bf1-b6ca-26e5daac3461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:51.794093 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 584/1155] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:51.816822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7cb06c-18d8-47f0-bfb3-e448136acff6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:51.925503 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7cb06c-18d8-47f0-bfb3-e448136acff6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:51.959575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384bcaa-a884-4ee0-b433-166d4ff1bc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:51.960143 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384bcaa-a884-4ee0-b433-166d4ff1bc01 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:51.960423 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384bcaa-a884-4ee0-b433-166d4ff1bc01 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:51.960695 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384bcaa-a884-4ee0-b433-166d4ff1bc01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:51.970922 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384bcaa-a884-4ee0-b433-166d4ff1bc01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:16:51.976948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-b384bcaa-a884-4ee0-b433-166d4ff1bc01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:16:51.976948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:51.976948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:51.976948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:51.976948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:51.976948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:51.976948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:51.976948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:51.976948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:51.976948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:51.976948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:51.976948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:51.976948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:16:51.976948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:16:51.976948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:16:51.976948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:16:51.978096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:16:51.978096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:16:51.978096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:16:51.978096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:16:51.978096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:51.978096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:51.978096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:51.978096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:51.978096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:51.978096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:51.978096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:51.978096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:51.978096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:51.978096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:51.978096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:51.978096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:51.979320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:51.979320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:51.979320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:51.979320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:51.979320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:16:51.979320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:16:51.979320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:16:51.979320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:16:51.979320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:16:51.979320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:16:51.979320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:16:51.979320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:16:51.979320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 572/1156] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:16:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:51.984069 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8f505-c8c3-4bf1-b6ca-26e5daac3461 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:52.004408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:52.083488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db8f505-c8c3-4bf1-b6ca-26e5daac3461 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:52.116090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6f20d-52e2-4cbb-b76b-91307a35292e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:52.116694 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6f20d-52e2-4cbb-b76b-91307a35292e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:52.116983 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6f20d-52e2-4cbb-b76b-91307a35292e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:52.117259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6f20d-52e2-4cbb-b76b-91307a35292e None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:52.118000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6f20d-52e2-4cbb-b76b-91307a35292e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:52.118308 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6f20d-52e2-4cbb-b76b-91307a35292e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:52.119675 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6f20d-52e2-4cbb-b76b-91307a35292e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:52.127449 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 585/1157] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:16:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:52.133806 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384bcaa-a884-4ee0-b433-166d4ff1bc01 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:52.137483 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:52.232379 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384bcaa-a884-4ee0-b433-166d4ff1bc01 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:52.265039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49dc60-a6e9-4e49-a76d-e94cd303fc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:52.265691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49dc60-a6e9-4e49-a76d-e94cd303fc74 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:52.265971 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49dc60-a6e9-4e49-a76d-e94cd303fc74 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:52.266242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49dc60-a6e9-4e49-a76d-e94cd303fc74 None admin] PATH_INFO: `/v3/users/3b562902ef404a5dbdb3287d4f82bfdb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:52.267386 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49dc60-a6e9-4e49-a76d-e94cd303fc74 None admin] RBAC: Authorizing `identity:get_user(user_id=3b562902ef404a5dbdb3287d4f82bfdb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:52.268815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49dc60-a6e9-4e49-a76d-e94cd303fc74 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:52.271003 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 573/1158] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:16:52 2022] GET /identity/v3/users/3b562902ef404a5dbdb3287d4f82bfdb => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:52.277396 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6f20d-52e2-4cbb-b76b-91307a35292e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:52.359492 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:52.378264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6f20d-52e2-4cbb-b76b-91307a35292e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:52.410348 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a22fe-841c-4277-9232-3b74e507315e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:52.411436 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a22fe-841c-4277-9232-3b74e507315e None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:52.411731 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a22fe-841c-4277-9232-3b74e507315e None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:52.412006 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a22fe-841c-4277-9232-3b74e507315e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:52.476776 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a22fe-841c-4277-9232-3b74e507315e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:52.478241 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a22fe-841c-4277-9232-3b74e507315e None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:52.555576 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:52.574613 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 586/1159] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:16:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 22:16:52.580340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49dc60-a6e9-4e49-a76d-e94cd303fc74 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:52.654408 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:52.684377 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49dc60-a6e9-4e49-a76d-e94cd303fc74 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:52.717010 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec545db-996e-4841-b6ab-d53367b3c457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:52.717669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec545db-996e-4841-b6ab-d53367b3c457 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:52.717952 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec545db-996e-4841-b6ab-d53367b3c457 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:52.718228 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec545db-996e-4841-b6ab-d53367b3c457 None admin] PATH_INFO: `/v3/projects/a2c261edaf464ce0872c694ae369019d` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:52.719373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec545db-996e-4841-b6ab-d53367b3c457 None admin] RBAC: Authorizing `identity:get_project(project_id=a2c261edaf464ce0872c694ae369019d)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:52.720811 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec545db-996e-4841-b6ab-d53367b3c457 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:52.722504 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 574/1160] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:16:52 2022] GET /identity/v3/projects/a2c261edaf464ce0872c694ae369019d => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:52.728626 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a22fe-841c-4277-9232-3b74e507315e None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:52.824848 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a22fe-841c-4277-9232-3b74e507315e None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:52.857991 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53043f-e4d6-448f-be09-decaa3755418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:52.858591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53043f-e4d6-448f-be09-decaa3755418 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:52.858875 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53043f-e4d6-448f-be09-decaa3755418 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:52.859152 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53043f-e4d6-448f-be09-decaa3755418 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:52.859934 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53043f-e4d6-448f-be09-decaa3755418 None admin] RBAC: Adding query filter params (role.id=aded7bd43e974686b3c23bd9d4255b24, scope.project.id=a2c261edaf464ce0872c694ae369019d, user.id=3b562902ef404a5dbdb3287d4f82bfdb) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:52.860245 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53043f-e4d6-448f-be09-decaa3755418 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:52.861699 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53043f-e4d6-448f-be09-decaa3755418 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:52.868028 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 587/1161] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:16:52 2022] GET /identity/v3/role_assignments?user.id=3b562902ef404a5dbdb3287d4f82bfdb&scope.project.id=a2c261edaf464ce0872c694ae369019d&role.id=aded7bd43e974686b3c23bd9d4255b24 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:54.922923 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d71580-6ae9-449d-856e-23d00e72f35a None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:54.923453 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d71580-6ae9-449d-856e-23d00e72f35a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:54.923453 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d71580-6ae9-449d-856e-23d00e72f35a None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:54.924815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 575/1162] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:54.930855 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fada0c-df60-47a1-a483-0285274b40d3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:54.931358 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fada0c-df60-47a1-a483-0285274b40d3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:54.931462 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fada0c-df60-47a1-a483-0285274b40d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:54.950163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a2fada0c-df60-47a1-a483-0285274b40d3 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:54.951330 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a2fada0c-df60-47a1-a483-0285274b40d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:54.991507 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:55.016532 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 588/1163] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:55.038502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d71580-6ae9-449d-856e-23d00e72f35a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:55.138973 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d71580-6ae9-449d-856e-23d00e72f35a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:55.185377 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba06c2-dddc-4e5e-a9ac-a25b3466428c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:55.185954 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba06c2-dddc-4e5e-a9ac-a25b3466428c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:55.186235 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba06c2-dddc-4e5e-a9ac-a25b3466428c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:55.186510 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ba06c2-dddc-4e5e-a9ac-a25b3466428c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:55.197196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:55.251736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba06c2-dddc-4e5e-a9ac-a25b3466428c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:55.253162 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ba06c2-dddc-4e5e-a9ac-a25b3466428c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:55.347713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 576/1164] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:16:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 22:16:55.353789 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fada0c-df60-47a1-a483-0285274b40d3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:55.448898 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fada0c-df60-47a1-a483-0285274b40d3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:55.481324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1e81a-7ffb-4caf-abb0-fd3c28dc3892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:55.481975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1e81a-7ffb-4caf-abb0-fd3c28dc3892 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:55.482253 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1e81a-7ffb-4caf-abb0-fd3c28dc3892 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:55.482524 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1e81a-7ffb-4caf-abb0-fd3c28dc3892 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:55.489643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1e81a-7ffb-4caf-abb0-fd3c28dc3892 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:55.491020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1e81a-7ffb-4caf-abb0-fd3c28dc3892 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:55.502138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-68e1e81a-7ffb-4caf-abb0-fd3c28dc3892 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 22:16:55.502138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:55.502138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 28 22:16:55.502138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 28 22:16:55.502138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:16:55.502138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:16:55.502138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:16:55.502138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:16:55.502138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 28 22:16:55.502138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:55.502138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:16:55.502138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:55.502138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:16:55.502138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:16:55.502138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:16:55.502138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:16:55.502138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:16:55.503410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:16:55.503410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:16:55.503410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:16:55.503410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:16:55.503410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:16:55.503410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:16:55.503410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:16:55.503410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:16:55.503410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:16:55.503410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:16:55.503410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:16:55.503410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:16:55.503410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:16:55.503410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:16:55.503410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:16:55.503410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:16:55.504591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:16:55.504591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:16:55.504591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:16:55.504591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:16:55.504591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:16:55.504591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:16:55.504591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:16:55.504591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:16:55.504591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:16:55.504591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:16:55.504591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 28 22:16:55.504591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:16:55.504591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 28 22:16:55.504591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:16:55.504591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 589/1165] 10.222.0.39 () {56 vars in 1117 bytes} [Thu Jul 28 22:16:55 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:16:55.509890 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba06c2-dddc-4e5e-a9ac-a25b3466428c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:55.549593 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:55.605416 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ba06c2-dddc-4e5e-a9ac-a25b3466428c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:55.638667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18acb165-4583-4138-b18e-55e1514430cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:55.639263 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acb165-4583-4138-b18e-55e1514430cb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:55.639540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acb165-4583-4138-b18e-55e1514430cb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:55.639811 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18acb165-4583-4138-b18e-55e1514430cb None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:55.640492 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18acb165-4583-4138-b18e-55e1514430cb None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:16:55.640790 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18acb165-4583-4138-b18e-55e1514430cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:55.642166 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18acb165-4583-4138-b18e-55e1514430cb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:55.649285 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 577/1166] 10.222.0.39 () {56 vars in 1115 bytes} [Thu Jul 28 22:16:55 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:16:57.612651 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b69450-e920-43a9-bc87-953dab76c464 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:57.613157 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b69450-e920-43a9-bc87-953dab76c464 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:57.613265 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b69450-e920-43a9-bc87-953dab76c464 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:57.615095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 590/1167] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:57.620601 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde315c-75e4-4a7a-9592-5959d9fb021b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:57.621036 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde315c-75e4-4a7a-9592-5959d9fb021b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:57.621177 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dde315c-75e4-4a7a-9592-5959d9fb021b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:57.639730 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9dde315c-75e4-4a7a-9592-5959d9fb021b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:57.640966 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9dde315c-75e4-4a7a-9592-5959d9fb021b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:57.708802 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 578/1168] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:16:57.733564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b69450-e920-43a9-bc87-953dab76c464 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:16:57.773860 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:16:57.842487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b69450-e920-43a9-bc87-953dab76c464 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:16:57.876383 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8b8826-edce-40bb-b8b3-c0b82035df2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:16:57.876933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b8826-edce-40bb-b8b3-c0b82035df2a None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:57.877240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b8826-edce-40bb-b8b3-c0b82035df2a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:57.877517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8b8826-edce-40bb-b8b3-c0b82035df2a None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:57.878261 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b8826-edce-40bb-b8b3-c0b82035df2a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:16:57.879728 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8b8826-edce-40bb-b8b3-c0b82035df2a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:16:57.895964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 591/1169] 10.222.0.39 () {60 vars in 1138 bytes} [Thu Jul 28 22:16:57 2022] POST /identity/v3/domains => generated 267 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:16:59.903630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161fd77-6683-48ca-9d92-223ca90f3d34 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:59.904182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161fd77-6683-48ca-9d92-223ca90f3d34 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:59.904182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b161fd77-6683-48ca-9d92-223ca90f3d34 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:59.905541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 579/1170] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:16:59.910539 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54610a-daf1-487c-ac94-b3f3444fb17f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:16:59.911393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54610a-daf1-487c-ac94-b3f3444fb17f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:16:59.911639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e54610a-daf1-487c-ac94-b3f3444fb17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:16:59.929282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-7e54610a-daf1-487c-ac94-b3f3444fb17f None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:16:59.930592 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7e54610a-daf1-487c-ac94-b3f3444fb17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:16:59.996127 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 592/1171] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:16:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:17:00.020333 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161fd77-6683-48ca-9d92-223ca90f3d34 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:00.055169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:17:00.120583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b161fd77-6683-48ca-9d92-223ca90f3d34 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:00.153844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef090e2f-77c2-4d68-8624-ba816d3f39c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:00.154419 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef090e2f-77c2-4d68-8624-ba816d3f39c6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:00.154697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef090e2f-77c2-4d68-8624-ba816d3f39c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:00.154969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef090e2f-77c2-4d68-8624-ba816d3f39c6 None admin] PATH_INFO: `/v3/domains/365aad6a566c4b3aaa6d9f352ad41240` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:00.165278 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef090e2f-77c2-4d68-8624-ba816d3f39c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=365aad6a566c4b3aaa6d9f352ad41240)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:17:00.166747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef090e2f-77c2-4d68-8624-ba816d3f39c6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:17:00.168336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 580/1172] 10.222.0.39 () {56 vars in 1183 bytes} [Thu Jul 28 22:17:00 2022] GET /identity/v3/domains/365aad6a566c4b3aaa6d9f352ad41240 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:17:00.174331 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54610a-daf1-487c-ac94-b3f3444fb17f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:00.201411 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:17:00.272809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e54610a-daf1-487c-ac94-b3f3444fb17f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:00.305097 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bf97bf-c6e7-4f8c-8203-a6ba5543130f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:00.305724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf97bf-c6e7-4f8c-8203-a6ba5543130f None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:00.306005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf97bf-c6e7-4f8c-8203-a6ba5543130f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:00.306276 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bf97bf-c6e7-4f8c-8203-a6ba5543130f None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:00.306960 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf97bf-c6e7-4f8c-8203-a6ba5543130f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:17:00.308370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bf97bf-c6e7-4f8c-8203-a6ba5543130f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:17:00.333880 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 593/1173] 10.222.0.39 () {60 vars in 1141 bytes} [Thu Jul 28 22:17:00 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:17:02.366852 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fdaaf-9fb0-4dca-845e-0960c257a09d None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:02.367403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fdaaf-9fb0-4dca-845e-0960c257a09d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:02.367403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1fdaaf-9fb0-4dca-845e-0960c257a09d None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:02.368732 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 581/1174] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:17:02.374085 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3673fa-2066-4b18-bd03-bfabcf202ef1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:02.374475 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3673fa-2066-4b18-bd03-bfabcf202ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:02.374637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3673fa-2066-4b18-bd03-bfabcf202ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:02.395218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ab3673fa-2066-4b18-bd03-bfabcf202ef1 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:17:02.396422 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ab3673fa-2066-4b18-bd03-bfabcf202ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:17:02.466867 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 594/1175] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:17:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:17:02.491853 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fdaaf-9fb0-4dca-845e-0960c257a09d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:02.583414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:17:02.591067 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1fdaaf-9fb0-4dca-845e-0960c257a09d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:02.623939 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05816662-9223-4b92-9afa-2b1c02e59ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:02.624573 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05816662-9223-4b92-9afa-2b1c02e59ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:02.624851 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05816662-9223-4b92-9afa-2b1c02e59ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:02.625123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05816662-9223-4b92-9afa-2b1c02e59ad5 None admin] PATH_INFO: `/v3/domains/365aad6a566c4b3aaa6d9f352ad41240` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:02.626296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05816662-9223-4b92-9afa-2b1c02e59ad5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=365aad6a566c4b3aaa6d9f352ad41240)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:17:02.627707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05816662-9223-4b92-9afa-2b1c02e59ad5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:17:02.629333 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 582/1176] 10.222.0.39 () {56 vars in 1183 bytes} [Thu Jul 28 22:17:02 2022] GET /identity/v3/domains/365aad6a566c4b3aaa6d9f352ad41240 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:17:02.635282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3673fa-2066-4b18-bd03-bfabcf202ef1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:02.646198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:17:02.737870 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3673fa-2066-4b18-bd03-bfabcf202ef1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:02.778654 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a5668f-0807-48cd-b647-7097906106d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:02.779274 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5668f-0807-48cd-b647-7097906106d1 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:02.779568 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5668f-0807-48cd-b647-7097906106d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:02.779848 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a5668f-0807-48cd-b647-7097906106d1 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:02.780547 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5668f-0807-48cd-b647-7097906106d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:17:02.782048 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a5668f-0807-48cd-b647-7097906106d1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:17:02.803884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 595/1177] 10.222.0.39 () {60 vars in 1132 bytes} [Thu Jul 28 22:17:02 2022] POST /identity/v3/users => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:17:04.823847 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af08c7-2de8-4c2c-bf5d-4d07569f5fa0 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:04.824424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af08c7-2de8-4c2c-bf5d-4d07569f5fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:04.824424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7af08c7-2de8-4c2c-bf5d-4d07569f5fa0 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:04.825792 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 583/1178] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:17:04.830974 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae89a4-b64a-4345-8620-6913cfb77885 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:04.831367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae89a4-b64a-4345-8620-6913cfb77885 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:04.831527 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae89a4-b64a-4345-8620-6913cfb77885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:04.849810 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9aae89a4-b64a-4345-8620-6913cfb77885 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:17:04.850983 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9aae89a4-b64a-4345-8620-6913cfb77885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:17:04.907960 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:17:04.924237 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 596/1179] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:17:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:17:04.948078 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af08c7-2de8-4c2c-bf5d-4d07569f5fa0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:05.050047 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7af08c7-2de8-4c2c-bf5d-4d07569f5fa0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:05.083289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7af82d-2fa0-43f2-9dd7-d0d136d904a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:05.083856 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7af82d-2fa0-43f2-9dd7-d0d136d904a6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:05.084139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7af82d-2fa0-43f2-9dd7-d0d136d904a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:05.084415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7af82d-2fa0-43f2-9dd7-d0d136d904a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:05.094731 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7af82d-2fa0-43f2-9dd7-d0d136d904a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:17:05.100254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-cd7af82d-2fa0-43f2-9dd7-d0d136d904a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:17:05.100254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:17:05.100254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:17:05.100254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:17:05.100254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:17:05.100254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:17:05.100254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:17:05.100254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:17:05.100254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:17:05.100254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:17:05.100254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:17:05.100254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:17:05.100254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:17:05.100254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:17:05.100254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:17:05.100254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:17:05.101367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:17:05.101367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:17:05.101367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:17:05.101367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:17:05.101367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:17:05.101367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:17:05.101367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:17:05.101367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:17:05.101367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:17:05.101367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:17:05.101367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:17:05.101367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:17:05.101367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:17:05.101367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:17:05.101367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:17:05.101367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:17:05.102465 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:17:05.102465 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:17:05.102465 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:17:05.102465 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:17:05.102465 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:17:05.102465 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:17:05.102465 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:17:05.102465 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:17:05.102465 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:17:05.102465 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:17:05.102465 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:17:05.102465 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:17:05.102465 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 584/1180] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:17:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:17:05.107476 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae89a4-b64a-4345-8620-6913cfb77885 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:05.203657 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae89a4-b64a-4345-8620-6913cfb77885 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:05.236593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52514cc7-48f1-4b39-92f3-cbf93a015ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:05.237198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52514cc7-48f1-4b39-92f3-cbf93a015ea6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:05.237475 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52514cc7-48f1-4b39-92f3-cbf93a015ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:05.237784 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52514cc7-48f1-4b39-92f3-cbf93a015ea6 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:05.238701 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52514cc7-48f1-4b39-92f3-cbf93a015ea6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:17:05.239415 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52514cc7-48f1-4b39-92f3-cbf93a015ea6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:17:05.243168 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52514cc7-48f1-4b39-92f3-cbf93a015ea6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:17:05.251151 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 597/1181] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:17:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:17:05.256960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7af82d-2fa0-43f2-9dd7-d0d136d904a6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:05.281966 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:17:05.354893 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7af82d-2fa0-43f2-9dd7-d0d136d904a6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:05.390700 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3310cbd-3297-4e68-b212-b2e338fa7a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:05.391335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3310cbd-3297-4e68-b212-b2e338fa7a6e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:05.391610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3310cbd-3297-4e68-b212-b2e338fa7a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:05.391877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3310cbd-3297-4e68-b212-b2e338fa7a6e None admin] PATH_INFO: `/v3/users/74030ff40e69423bac67620373b699e9` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:05.405446 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3310cbd-3297-4e68-b212-b2e338fa7a6e None admin] RBAC: Authorizing `identity:get_user(user_id=74030ff40e69423bac67620373b699e9)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:17:05.406895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3310cbd-3297-4e68-b212-b2e338fa7a6e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:17:05.408483 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 585/1182] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:17:05 2022] GET /identity/v3/users/74030ff40e69423bac67620373b699e9 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:17:05.414959 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52514cc7-48f1-4b39-92f3-cbf93a015ea6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:05.469401 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:17:05.513169 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52514cc7-48f1-4b39-92f3-cbf93a015ea6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:05.545493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eaf404-240c-44f1-b1f6-32aeaaafe5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:05.546124 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eaf404-240c-44f1-b1f6-32aeaaafe5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:05.546409 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eaf404-240c-44f1-b1f6-32aeaaafe5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:05.546684 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eaf404-240c-44f1-b1f6-32aeaaafe5b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:05.611344 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eaf404-240c-44f1-b1f6-32aeaaafe5b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:17:05.612725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4eaf404-240c-44f1-b1f6-32aeaaafe5b2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:17:05.665305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:17:05.708773 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 598/1183] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:17:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 22:17:05.714543 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3310cbd-3297-4e68-b212-b2e338fa7a6e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:05.795778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:17:05.816118 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3310cbd-3297-4e68-b212-b2e338fa7a6e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:05.848203 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbc2ca7-b6cb-42d7-990d-b9a2bf3d0a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:05.849508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc2ca7-b6cb-42d7-990d-b9a2bf3d0a9b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:05.849918 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc2ca7-b6cb-42d7-990d-b9a2bf3d0a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:05.850195 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbc2ca7-b6cb-42d7-990d-b9a2bf3d0a9b None admin] PATH_INFO: `/v3/projects/1e1992384cab4d709b341fd44c281607` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:05.851335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc2ca7-b6cb-42d7-990d-b9a2bf3d0a9b None admin] RBAC: Authorizing `identity:get_project(project_id=1e1992384cab4d709b341fd44c281607)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:17:05.852754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbc2ca7-b6cb-42d7-990d-b9a2bf3d0a9b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:17:05.854615 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 586/1184] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:17:05 2022] GET /identity/v3/projects/1e1992384cab4d709b341fd44c281607 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:17:05.860526 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eaf404-240c-44f1-b1f6-32aeaaafe5b2 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:05.956318 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eaf404-240c-44f1-b1f6-32aeaaafe5b2 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:05.990363 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295aa61-002f-4ce6-98b5-c2039a2a6c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:05.990966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295aa61-002f-4ce6-98b5-c2039a2a6c4a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:05.991249 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295aa61-002f-4ce6-98b5-c2039a2a6c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:05.991524 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295aa61-002f-4ce6-98b5-c2039a2a6c4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:05.992310 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295aa61-002f-4ce6-98b5-c2039a2a6c4a None admin] RBAC: Adding query filter params (role.id=aded7bd43e974686b3c23bd9d4255b24, scope.project.id=1e1992384cab4d709b341fd44c281607, user.id=74030ff40e69423bac67620373b699e9) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:17:05.992623 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295aa61-002f-4ce6-98b5-c2039a2a6c4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:17:05.994177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295aa61-002f-4ce6-98b5-c2039a2a6c4a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:17:05.999443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 599/1185] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:17:05 2022] GET /identity/v3/role_assignments?user.id=74030ff40e69423bac67620373b699e9&scope.project.id=1e1992384cab4d709b341fd44c281607&role.id=aded7bd43e974686b3c23bd9d4255b24 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:17:07.959735 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8929aec-ff63-4e68-a978-34a9da39fe76 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:07.960288 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8929aec-ff63-4e68-a978-34a9da39fe76 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:07.960288 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8929aec-ff63-4e68-a978-34a9da39fe76 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:07.961658 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 587/1186] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:17:07.966839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa226ba-be42-4398-ab9e-2c0eda1bab95 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:07.967234 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa226ba-be42-4398-ab9e-2c0eda1bab95 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:07.967401 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa226ba-be42-4398-ab9e-2c0eda1bab95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:07.985625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8fa226ba-be42-4398-ab9e-2c0eda1bab95 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:17:07.986813 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8fa226ba-be42-4398-ab9e-2c0eda1bab95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:17:08.001148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:17:08.053675 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 600/1187] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:17:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 28 22:17:08.076158 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8929aec-ff63-4e68-a978-34a9da39fe76 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:08.176469 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8929aec-ff63-4e68-a978-34a9da39fe76 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:08.210041 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c60d3-a88a-4b31-9b5a-d1d776524f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:08.210614 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c60d3-a88a-4b31-9b5a-d1d776524f41 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:08.210896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c60d3-a88a-4b31-9b5a-d1d776524f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:08.211170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c60d3-a88a-4b31-9b5a-d1d776524f41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:08.221001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c60d3-a88a-4b31-9b5a-d1d776524f41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:17:08.226644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-b40c60d3-a88a-4b31-9b5a-d1d776524f41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:17:08.226644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:17:08.226644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:17:08.226644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:17:08.226644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:17:08.226644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:17:08.226644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:17:08.226644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:17:08.226644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:17:08.226644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:17:08.226644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:17:08.226644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:17:08.226644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:17:08.226644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:17:08.226644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:17:08.226644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:17:08.227809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:17:08.227809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:17:08.227809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:17:08.227809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:17:08.227809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:17:08.227809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:17:08.227809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:17:08.227809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:17:08.227809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:17:08.227809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:17:08.227809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:17:08.227809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:17:08.227809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:17:08.227809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:17:08.227809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:17:08.227809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:17:08.228888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:17:08.228888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:17:08.228888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:17:08.228888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:17:08.228888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:17:08.228888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:17:08.228888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:17:08.228888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:17:08.228888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:17:08.228888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:17:08.228888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:17:08.228888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:17:08.228888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 588/1188] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:17:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:17:08.233763 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa226ba-be42-4398-ab9e-2c0eda1bab95 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:08.330522 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa226ba-be42-4398-ab9e-2c0eda1bab95 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:08.364395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22674b6-2a81-477e-8b94-b78e0db2edd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:08.365003 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22674b6-2a81-477e-8b94-b78e0db2edd4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:08.365283 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22674b6-2a81-477e-8b94-b78e0db2edd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:08.365583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22674b6-2a81-477e-8b94-b78e0db2edd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:08.366294 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22674b6-2a81-477e-8b94-b78e0db2edd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:17:08.366601 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22674b6-2a81-477e-8b94-b78e0db2edd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:17:08.367967 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22674b6-2a81-477e-8b94-b78e0db2edd4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:17:08.375835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 601/1189] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:17:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:17:08.381450 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c60d3-a88a-4b31-9b5a-d1d776524f41 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:08.392981 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:17:08.479876 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c60d3-a88a-4b31-9b5a-d1d776524f41 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:08.512951 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987382b6-caac-4928-8d1e-8f6186f4722c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:08.513586 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987382b6-caac-4928-8d1e-8f6186f4722c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:08.513867 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987382b6-caac-4928-8d1e-8f6186f4722c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:08.514133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987382b6-caac-4928-8d1e-8f6186f4722c None admin] PATH_INFO: `/v3/users/74030ff40e69423bac67620373b699e9` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:08.515274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987382b6-caac-4928-8d1e-8f6186f4722c None admin] RBAC: Authorizing `identity:get_user(user_id=74030ff40e69423bac67620373b699e9)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:17:08.516677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987382b6-caac-4928-8d1e-8f6186f4722c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:17:08.518260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 589/1190] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:17:08 2022] GET /identity/v3/users/74030ff40e69423bac67620373b699e9 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:17:08.524747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22674b6-2a81-477e-8b94-b78e0db2edd4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:08.540250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:17:08.623896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22674b6-2a81-477e-8b94-b78e0db2edd4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:08.656324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b14b87-afec-43b0-931d-b36c96c087e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:08.656956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b14b87-afec-43b0-931d-b36c96c087e2 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:08.657242 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b14b87-afec-43b0-931d-b36c96c087e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:08.657546 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b14b87-afec-43b0-931d-b36c96c087e2 None admin] PATH_INFO: `/v3/projects/1e1992384cab4d709b341fd44c281607` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:08.658716 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b14b87-afec-43b0-931d-b36c96c087e2 None admin] RBAC: Authorizing `identity:get_project(project_id=1e1992384cab4d709b341fd44c281607)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:17:08.660144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b14b87-afec-43b0-931d-b36c96c087e2 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:17:08.662563 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 602/1191] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:17:08 2022] GET /identity/v3/projects/1e1992384cab4d709b341fd44c281607 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:17:08.668422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987382b6-caac-4928-8d1e-8f6186f4722c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:08.742772 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:17:08.765419 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987382b6-caac-4928-8d1e-8f6186f4722c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:08.801112 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e58da5-dbb8-4993-8ef3-6461ae6106d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:08.802386 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e58da5-dbb8-4993-8ef3-6461ae6106d3 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:08.802665 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e58da5-dbb8-4993-8ef3-6461ae6106d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:08.802932 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e58da5-dbb8-4993-8ef3-6461ae6106d3 None admin] PATH_INFO: `/v3/projects/1e1992384cab4d709b341fd44c281607/users/74030ff40e69423bac67620373b699e9/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:08.804826 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e58da5-dbb8-4993-8ef3-6461ae6106d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e1992384cab4d709b341fd44c281607, user_id=74030ff40e69423bac67620373b699e9, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:17:08.806295 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e58da5-dbb8-4993-8ef3-6461ae6106d3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:17:08.818315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 590/1192] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:17:08 2022] PUT /identity/v3/projects/1e1992384cab4d709b341fd44c281607/users/74030ff40e69423bac67620373b699e9/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:17:10.801667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f749b-aab4-4067-923f-3a8302c36ecd None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:10.802229 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f749b-aab4-4067-923f-3a8302c36ecd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:10.802229 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1f749b-aab4-4067-923f-3a8302c36ecd None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:10.803619 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 603/1193] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:17:10.808944 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8b2f5-69fa-4b0d-97b3-421fbc59c0be None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:10.809332 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8b2f5-69fa-4b0d-97b3-421fbc59c0be None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:10.809495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8b2f5-69fa-4b0d-97b3-421fbc59c0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:10.827859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-afa8b2f5-69fa-4b0d-97b3-421fbc59c0be None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:17:10.829047 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-afa8b2f5-69fa-4b0d-97b3-421fbc59c0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:17:10.894797 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 591/1194] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:17:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:17:10.918506 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f749b-aab4-4067-923f-3a8302c36ecd None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:10.999025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:17:11.019330 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1f749b-aab4-4067-923f-3a8302c36ecd None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:11.051652 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b80c6-6fa1-4bda-90de-568149fb89ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:11.052323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b80c6-6fa1-4bda-90de-568149fb89ed None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:11.052554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b80c6-6fa1-4bda-90de-568149fb89ed None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:11.052825 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b80c6-6fa1-4bda-90de-568149fb89ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:11.062443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b80c6-6fa1-4bda-90de-568149fb89ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:17:11.067852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-025b80c6-6fa1-4bda-90de-568149fb89ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:17:11.067852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:17:11.067852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:17:11.067852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:17:11.067852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:17:11.067852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:17:11.067852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:17:11.067852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:17:11.067852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:17:11.067852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:17:11.067852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:17:11.067852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:17:11.067852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:17:11.067852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:17:11.067852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:17:11.067852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:17:11.069002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:17:11.069002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:17:11.069002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:17:11.069002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:17:11.069002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:17:11.069002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:17:11.069002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:17:11.069002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:17:11.069002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:17:11.069002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:17:11.069002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:17:11.069002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:17:11.069002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:17:11.069002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:17:11.069002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:17:11.069002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:17:11.070272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:17:11.070272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:17:11.070272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:17:11.070272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:17:11.070272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:17:11.070272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:17:11.070272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:17:11.070272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:17:11.070272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:17:11.070272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:17:11.070272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:17:11.070272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:17:11.070272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 604/1195] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:17:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:17:11.075114 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8b2f5-69fa-4b0d-97b3-421fbc59c0be None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:11.139136 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:17:11.174268 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8b2f5-69fa-4b0d-97b3-421fbc59c0be None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:11.208755 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e91e5-1f4b-4588-9d6c-af935ba76759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:11.209372 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e91e5-1f4b-4588-9d6c-af935ba76759 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:11.209691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e91e5-1f4b-4588-9d6c-af935ba76759 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:11.209970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e91e5-1f4b-4588-9d6c-af935ba76759 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:11.210683 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1e91e5-1f4b-4588-9d6c-af935ba76759 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:17:11.210987 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1e91e5-1f4b-4588-9d6c-af935ba76759 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:17:11.212355 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1e91e5-1f4b-4588-9d6c-af935ba76759 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:17:11.220208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 592/1196] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:17:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:17:11.226282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b80c6-6fa1-4bda-90de-568149fb89ed None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:11.324371 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b80c6-6fa1-4bda-90de-568149fb89ed None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:11.356554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a196d7-78cd-4a2c-b1f3-67c517775472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:11.357170 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a196d7-78cd-4a2c-b1f3-67c517775472 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:11.357454 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a196d7-78cd-4a2c-b1f3-67c517775472 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:11.357755 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a196d7-78cd-4a2c-b1f3-67c517775472 None admin] PATH_INFO: `/v3/users/74030ff40e69423bac67620373b699e9` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:11.358922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a196d7-78cd-4a2c-b1f3-67c517775472 None admin] RBAC: Authorizing `identity:get_user(user_id=74030ff40e69423bac67620373b699e9)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:17:11.360341 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a196d7-78cd-4a2c-b1f3-67c517775472 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:17:11.361969 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 605/1197] 10.222.0.39 () {56 vars in 1177 bytes} [Thu Jul 28 22:17:11 2022] GET /identity/v3/users/74030ff40e69423bac67620373b699e9 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:17:11.368120 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e91e5-1f4b-4588-9d6c-af935ba76759 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:11.465258 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e91e5-1f4b-4588-9d6c-af935ba76759 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:11.490620 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:17:11.498596 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cf971-3be5-4cd2-a0e0-7c3f892e75ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:11.499200 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cf971-3be5-4cd2-a0e0-7c3f892e75ee None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:11.499481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cf971-3be5-4cd2-a0e0-7c3f892e75ee None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:11.499753 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cf971-3be5-4cd2-a0e0-7c3f892e75ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:11.565392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5cf971-3be5-4cd2-a0e0-7c3f892e75ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:17:11.566857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5cf971-3be5-4cd2-a0e0-7c3f892e75ee None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:17:11.661761 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 593/1198] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:17:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 22:17:11.667510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a196d7-78cd-4a2c-b1f3-67c517775472 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:11.679074 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:17:11.765622 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a196d7-78cd-4a2c-b1f3-67c517775472 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:11.798364 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1de55c-dc67-4b2f-884b-b621631e981f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:11.798987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1de55c-dc67-4b2f-884b-b621631e981f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:11.799269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1de55c-dc67-4b2f-884b-b621631e981f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:11.799542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1de55c-dc67-4b2f-884b-b621631e981f None admin] PATH_INFO: `/v3/projects/1e1992384cab4d709b341fd44c281607` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:11.800681 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1de55c-dc67-4b2f-884b-b621631e981f None admin] RBAC: Authorizing `identity:get_project(project_id=1e1992384cab4d709b341fd44c281607)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:17:11.802714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1de55c-dc67-4b2f-884b-b621631e981f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:17:11.804368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 606/1199] 10.222.0.39 () {56 vars in 1186 bytes} [Thu Jul 28 22:17:11 2022] GET /identity/v3/projects/1e1992384cab4d709b341fd44c281607 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:17:11.810465 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cf971-3be5-4cd2-a0e0-7c3f892e75ee None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:17:11.837376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:17:11.907350 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cf971-3be5-4cd2-a0e0-7c3f892e75ee None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:17:11.940792 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2709e390-34d7-4481-bd6b-60386a9c087c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:17:11.941336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709e390-34d7-4481-bd6b-60386a9c087c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:17:11.941649 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709e390-34d7-4481-bd6b-60386a9c087c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:17:11.941920 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2709e390-34d7-4481-bd6b-60386a9c087c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:17:11.942741 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2709e390-34d7-4481-bd6b-60386a9c087c None admin] RBAC: Adding query filter params (role.id=aded7bd43e974686b3c23bd9d4255b24, scope.project.id=1e1992384cab4d709b341fd44c281607, user.id=74030ff40e69423bac67620373b699e9) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:17:11.943046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2709e390-34d7-4481-bd6b-60386a9c087c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:17:11.944466 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2709e390-34d7-4481-bd6b-60386a9c087c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:17:11.950118 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 594/1200] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:17:11 2022] GET /identity/v3/role_assignments?user.id=74030ff40e69423bac67620373b699e9&scope.project.id=1e1992384cab4d709b341fd44c281607&role.id=aded7bd43e974686b3c23bd9d4255b24 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:17:21.794260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=62 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:17:21.947592 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=54 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:07.441643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b4a11-1b7e-4d13-90ea-245831a38155 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:07.442481 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b4a11-1b7e-4d13-90ea-245831a38155 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:07.442771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b4a11-1b7e-4d13-90ea-245831a38155 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:07.444161 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 607/1201] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:18:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:18:07.449935 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b4f75-142c-42ce-acf9-4cb2dd405926 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:07.450379 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b4f75-142c-42ce-acf9-4cb2dd405926 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:07.453278 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b4f75-142c-42ce-acf9-4cb2dd405926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:07.471567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c78b4f75-142c-42ce-acf9-4cb2dd405926 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:18:07.473062 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c78b4f75-142c-42ce-acf9-4cb2dd405926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:18:07.570998 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 595/1202] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:18:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:18:07.590704 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b4a11-1b7e-4d13-90ea-245831a38155 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:07.721885 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b4a11-1b7e-4d13-90ea-245831a38155 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:07.767118 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907079b-9b63-42ab-ae37-195bf6ac997a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:07.768596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907079b-9b63-42ab-ae37-195bf6ac997a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:07.768903 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907079b-9b63-42ab-ae37-195bf6ac997a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:07.769187 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c907079b-9b63-42ab-ae37-195bf6ac997a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:07.782205 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907079b-9b63-42ab-ae37-195bf6ac997a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:07.784374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c907079b-9b63-42ab-ae37-195bf6ac997a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:07.805730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-c907079b-9b63-42ab-ae37-195bf6ac997a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:07.805730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:07.805730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:07.805730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:07.805730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:07.805730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:07.805730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:07.805730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:07.805730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:07.805730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:07.805730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:07.805730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:07.805730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:18:07.805730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:18:07.805730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:18:07.807189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:18:07.807189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:07.807189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:07.807189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:07.807189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:07.807189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:07.807189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:07.807189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:07.807189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:07.807189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:07.807189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:07.807189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:07.807189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:07.807189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:07.807189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:07.807189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:07.807189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:07.808230 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:18:07.808230 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:18:07.808230 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:07.808230 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:18:07.808230 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 608/1203] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:18:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:07.813674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b4f75-142c-42ce-acf9-4cb2dd405926 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:07.956118 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b4f75-142c-42ce-acf9-4cb2dd405926 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:08.004629 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:08.012571 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892b40c-321b-4541-8350-17e220ec8c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:08.013274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892b40c-321b-4541-8350-17e220ec8c33 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:08.013584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892b40c-321b-4541-8350-17e220ec8c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:08.013989 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892b40c-321b-4541-8350-17e220ec8c33 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:08.015002 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0892b40c-321b-4541-8350-17e220ec8c33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:08.015343 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0892b40c-321b-4541-8350-17e220ec8c33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:08.016964 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0892b40c-321b-4541-8350-17e220ec8c33 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:08.040707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 596/1204] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:18:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:08.049603 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907079b-9b63-42ab-ae37-195bf6ac997a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:08.132538 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:08.207690 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c907079b-9b63-42ab-ae37-195bf6ac997a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:08.244329 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300389e6-dfd0-48b8-b71e-7d936f2bf0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:08.245003 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300389e6-dfd0-48b8-b71e-7d936f2bf0e7 None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:08.245284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300389e6-dfd0-48b8-b71e-7d936f2bf0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:08.245590 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300389e6-dfd0-48b8-b71e-7d936f2bf0e7 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:08.246302 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300389e6-dfd0-48b8-b71e-7d936f2bf0e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:08.247737 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300389e6-dfd0-48b8-b71e-7d936f2bf0e7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:08.277636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 609/1205] 10.222.0.39 () {60 vars in 1132 bytes} [Thu Jul 28 22:18:08 2022] POST /identity/v3/users => generated 251 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:18:10.590825 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1facc-b9d2-490a-bb3c-3f4fac79a895 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:10.591370 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1facc-b9d2-490a-bb3c-3f4fac79a895 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:10.591370 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c1facc-b9d2-490a-bb3c-3f4fac79a895 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:10.593114 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 597/1206] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:18:10.599121 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1befdb-35cf-4829-8ebe-de7691c1dae1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:10.599633 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1befdb-35cf-4829-8ebe-de7691c1dae1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:10.599735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1befdb-35cf-4829-8ebe-de7691c1dae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:10.620981 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9d1befdb-35cf-4829-8ebe-de7691c1dae1 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:18:10.622314 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9d1befdb-35cf-4829-8ebe-de7691c1dae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:18:10.707258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 610/1207] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:18:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:18:10.732347 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1facc-b9d2-490a-bb3c-3f4fac79a895 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:10.858197 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c1facc-b9d2-490a-bb3c-3f4fac79a895 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:10.899411 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ca144-200c-4a5e-aa2f-d5ae82632da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:10.900154 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ca144-200c-4a5e-aa2f-d5ae82632da7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:10.900437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ca144-200c-4a5e-aa2f-d5ae82632da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:10.901119 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ca144-200c-4a5e-aa2f-d5ae82632da7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:10.917925 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ca144-200c-4a5e-aa2f-d5ae82632da7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:18:10.923043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-783ca144-200c-4a5e-aa2f-d5ae82632da7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:18:10.923043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:10.923043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:10.923043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:10.923043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:10.923043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:10.923043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:10.923043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:10.923043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:10.923043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:10.923043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:10.923043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:10.923043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:18:10.923043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:18:10.923043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:18:10.923043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:18:10.925055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:18:10.925055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:18:10.925055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:18:10.925055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:18:10.925055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:10.925055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:10.925055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:10.925055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:10.925055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:10.925055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:10.925055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:10.925055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:10.925055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:10.925055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:10.925055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:10.925055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:10.926257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:10.926257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:10.926257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:10.926257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:10.926257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:18:10.926257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:18:10.926257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:18:10.926257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:18:10.926257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:18:10.926257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:18:10.926257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:18:10.926257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:10.930119 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 598/1208] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:18:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:18:10.938706 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1befdb-35cf-4829-8ebe-de7691c1dae1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:10.984794 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:11.059866 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1befdb-35cf-4829-8ebe-de7691c1dae1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:11.103343 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235ac9e-828b-4c17-a995-00087730bd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:11.104130 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235ac9e-828b-4c17-a995-00087730bd63 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:11.104402 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235ac9e-828b-4c17-a995-00087730bd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:11.104675 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f235ac9e-828b-4c17-a995-00087730bd63 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:11.105597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235ac9e-828b-4c17-a995-00087730bd63 None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:11.105946 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235ac9e-828b-4c17-a995-00087730bd63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:11.107483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f235ac9e-828b-4c17-a995-00087730bd63 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:11.120844 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 611/1209] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:18:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:11.126707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ca144-200c-4a5e-aa2f-d5ae82632da7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:11.200047 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:11.267313 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ca144-200c-4a5e-aa2f-d5ae82632da7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:11.312025 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a63e11-72ea-405f-a63d-18d2ccadeaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:11.312822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a63e11-72ea-405f-a63d-18d2ccadeaa3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:11.313326 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a63e11-72ea-405f-a63d-18d2ccadeaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:11.313637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a63e11-72ea-405f-a63d-18d2ccadeaa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:11.327790 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a63e11-72ea-405f-a63d-18d2ccadeaa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:11.329598 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a63e11-72ea-405f-a63d-18d2ccadeaa3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:11.347406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c9a63e11-72ea-405f-a63d-18d2ccadeaa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:11.347406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:11.347406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:11.347406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:11.347406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:11.347406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:11.347406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:11.347406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:11.347406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:11.347406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:11.347406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:11.347406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:11.347406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:18:11.347406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:18:11.347406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:18:11.348404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:18:11.348404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:11.348404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:11.348404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:11.348404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:11.348404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:11.348404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:11.348404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:11.348404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:11.348404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:11.348404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:11.348404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:11.348404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:11.348404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:11.348404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:11.348404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:11.348404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:11.354451 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:18:11.354451 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:18:11.354451 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:11.354451 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:11.354451 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 599/1210] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:18:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:11.354817 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235ac9e-828b-4c17-a995-00087730bd63 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:11.488492 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:11.492665 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f235ac9e-828b-4c17-a995-00087730bd63 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:11.544556 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb450907-6ab9-483b-8178-cfa587312f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:11.545385 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb450907-6ab9-483b-8178-cfa587312f1a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:11.545750 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb450907-6ab9-483b-8178-cfa587312f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:11.546031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb450907-6ab9-483b-8178-cfa587312f1a None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:11.547137 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb450907-6ab9-483b-8178-cfa587312f1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:11.547446 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb450907-6ab9-483b-8178-cfa587312f1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:11.553382 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb450907-6ab9-483b-8178-cfa587312f1a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:11.566766 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 612/1211] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:11.573123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a63e11-72ea-405f-a63d-18d2ccadeaa3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:11.696326 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a63e11-72ea-405f-a63d-18d2ccadeaa3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:11.698199 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:11.743307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b84a3-f23a-4dca-b6ab-247ab97d5aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:11.743860 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b84a3-f23a-4dca-b6ab-247ab97d5aee None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:11.744137 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b84a3-f23a-4dca-b6ab-247ab97d5aee None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:11.744576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b84a3-f23a-4dca-b6ab-247ab97d5aee None admin] PATH_INFO: `/v3/users/placement` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:11.753516 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b84a3-f23a-4dca-b6ab-247ab97d5aee None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:11.754987 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848b84a3-f23a-4dca-b6ab-247ab97d5aee None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:11.766978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-848b84a3-f23a-4dca-b6ab-247ab97d5aee None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 22:18:11.766978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:11.766978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:11.766978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:11.766978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:11.766978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:11.766978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:11.766978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:11.766978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:11.766978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:11.766978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:11.766978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:11.766978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:18:11.766978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:18:11.766978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:18:11.766978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:18:11.768250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:11.768250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:11.768250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:18:11.768250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:18:11.768250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:18:11.768250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:18:11.768250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:11.768250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:11.768250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:11.768250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:11.768250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:11.768250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:11.768250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:11.768250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:11.768250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:11.768250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:11.768250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:11.768250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:11.769877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:11.769877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:11.769877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:18:11.769877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:18:11.769877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:18:11.769877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:18:11.769877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:18:11.769877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:18:11.769877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 22:18:11.769877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:11.769877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 600/1212] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:18:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:11.774170 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb450907-6ab9-483b-8178-cfa587312f1a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:11.907286 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb450907-6ab9-483b-8178-cfa587312f1a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:11.953962 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aac26d-0d5b-4d21-867e-df4314978d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:11.955429 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aac26d-0d5b-4d21-867e-df4314978d9c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:11.955885 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aac26d-0d5b-4d21-867e-df4314978d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:11.956162 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77aac26d-0d5b-4d21-867e-df4314978d9c None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:11.956959 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aac26d-0d5b-4d21-867e-df4314978d9c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:11.957264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aac26d-0d5b-4d21-867e-df4314978d9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:11.958720 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77aac26d-0d5b-4d21-867e-df4314978d9c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:11.985779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 613/1213] 10.222.0.39 () {56 vars in 1143 bytes} [Thu Jul 28 22:18:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:11.987572 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b84a3-f23a-4dca-b6ab-247ab97d5aee None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:12.121294 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b84a3-f23a-4dca-b6ab-247ab97d5aee None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:12.173974 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010af66-0d06-4298-a10d-d9ebdeae4907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:12.174545 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010af66-0d06-4298-a10d-d9ebdeae4907 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:12.174945 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010af66-0d06-4298-a10d-d9ebdeae4907 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:12.175324 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010af66-0d06-4298-a10d-d9ebdeae4907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:12.210525 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:12.260448 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010af66-0d06-4298-a10d-d9ebdeae4907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:12.263108 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010af66-0d06-4298-a10d-d9ebdeae4907 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:12.385231 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 601/1214] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:18:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 22:18:12.392467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aac26d-0d5b-4d21-867e-df4314978d9c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:12.418489 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:12.517153 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77aac26d-0d5b-4d21-867e-df4314978d9c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:12.555817 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc964279-b6b8-4573-99af-53343caed34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:12.556483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc964279-b6b8-4573-99af-53343caed34f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:12.556772 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc964279-b6b8-4573-99af-53343caed34f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:12.557113 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc964279-b6b8-4573-99af-53343caed34f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:12.567708 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc964279-b6b8-4573-99af-53343caed34f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:12.569155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc964279-b6b8-4573-99af-53343caed34f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:12.584618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-cc964279-b6b8-4573-99af-53343caed34f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:12.584618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:12.584618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:12.584618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:12.584618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:12.584618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:12.584618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:12.584618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:12.584618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:12.584618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:12.584618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:12.584618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:12.584618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:18:12.584618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:18:12.584618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:18:12.585659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:18:12.585659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:12.585659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:12.585659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:12.585659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:12.585659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:12.585659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:12.585659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:12.585659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:12.585659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:12.585659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:12.585659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:12.585659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:12.585659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:12.585659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:12.585659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:12.585659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:12.586595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:18:12.586595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:18:12.586595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:12.586595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:18:12.586595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 614/1215] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:18:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:18:12.591919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010af66-0d06-4298-a10d-d9ebdeae4907 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:12.671782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:12.714820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010af66-0d06-4298-a10d-d9ebdeae4907 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:12.755690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3d821-b89e-406b-9f26-e060b57e37fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:12.756282 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3d821-b89e-406b-9f26-e060b57e37fd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:12.756652 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3d821-b89e-406b-9f26-e060b57e37fd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:12.756843 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3d821-b89e-406b-9f26-e060b57e37fd None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:12.757538 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a3d821-b89e-406b-9f26-e060b57e37fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:12.757874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a3d821-b89e-406b-9f26-e060b57e37fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:12.759295 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a3d821-b89e-406b-9f26-e060b57e37fd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:12.772367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 602/1216] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:12.778136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc964279-b6b8-4573-99af-53343caed34f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:12.884753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:12.911800 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc964279-b6b8-4573-99af-53343caed34f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:12.948715 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69473e-65d3-45cf-be63-7d10651c2f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:12.949630 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69473e-65d3-45cf-be63-7d10651c2f78 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:12.950067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69473e-65d3-45cf-be63-7d10651c2f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:12.950487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69473e-65d3-45cf-be63-7d10651c2f78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:12.959670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f69473e-65d3-45cf-be63-7d10651c2f78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:12.962048 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f69473e-65d3-45cf-be63-7d10651c2f78 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:12.979710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-3f69473e-65d3-45cf-be63-7d10651c2f78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:18:12.979710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:12.979710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:12.979710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:12.979710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:12.979710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:12.979710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:12.979710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:12.979710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:12.979710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:12.979710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:12.979710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:12.979710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:18:12.979710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:18:12.979710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:18:12.981061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:18:12.981061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:12.981061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:12.981061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:12.981061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:12.981061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:12.981061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:12.981061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:12.981061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:12.981061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:12.981061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:12.981061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:12.981061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:12.981061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:12.981061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:12.981061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:12.981061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:12.982264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:18:12.982264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:18:12.982264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:18:12.982264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:18:12.982264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:18:12.982264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:18:12.982264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:18:12.982264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:18:12.985004 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 615/1217] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:18:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:12.990833 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3d821-b89e-406b-9f26-e060b57e37fd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:13.110799 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3d821-b89e-406b-9f26-e060b57e37fd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:13.142086 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:13.160521 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc88bf-d61b-4554-b445-76b5d5105633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:13.161137 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc88bf-d61b-4554-b445-76b5d5105633 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:13.161444 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc88bf-d61b-4554-b445-76b5d5105633 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:13.161725 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc88bf-d61b-4554-b445-76b5d5105633 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:13.162493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc88bf-d61b-4554-b445-76b5d5105633 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:13.162799 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc88bf-d61b-4554-b445-76b5d5105633 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:13.164217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc88bf-d61b-4554-b445-76b5d5105633 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:13.179097 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 603/1218] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:18:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:13.186417 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69473e-65d3-45cf-be63-7d10651c2f78 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:13.310765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69473e-65d3-45cf-be63-7d10651c2f78 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:13.344146 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:13.352082 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8836c49c-f8cf-4592-bfee-ecf08f29b530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:13.352700 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836c49c-f8cf-4592-bfee-ecf08f29b530 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:13.352980 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836c49c-f8cf-4592-bfee-ecf08f29b530 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:13.353251 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8836c49c-f8cf-4592-bfee-ecf08f29b530 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:13.354126 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836c49c-f8cf-4592-bfee-ecf08f29b530 None admin] RBAC: Adding query filter params (role.id=a66ab77a63174592a1e468d2464e8bc4, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=fe003cd7bfbe4b92b1d64d300c3e7d8b) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:13.354441 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836c49c-f8cf-4592-bfee-ecf08f29b530 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:13.355904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8836c49c-f8cf-4592-bfee-ecf08f29b530 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:13.373814 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 616/1219] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:18:13 2022] GET /identity/v3/role_assignments?user.id=fe003cd7bfbe4b92b1d64d300c3e7d8b&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=a66ab77a63174592a1e468d2464e8bc4 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:15.515616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d9cbb-b760-4129-b642-026abad804fe None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:15.516209 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d9cbb-b760-4129-b642-026abad804fe None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:15.516209 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d9cbb-b760-4129-b642-026abad804fe None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:15.518327 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 604/1220] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:18:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:18:15.524584 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c98a4e-23b4-4e0c-b545-5c31ba06084f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:15.525054 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c98a4e-23b4-4e0c-b545-5c31ba06084f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:15.525054 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c98a4e-23b4-4e0c-b545-5c31ba06084f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:15.550474 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b6c98a4e-23b4-4e0c-b545-5c31ba06084f None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:18:15.550474 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b6c98a4e-23b4-4e0c-b545-5c31ba06084f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:18:15.625555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 617/1221] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:18:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:18:15.647924 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d9cbb-b760-4129-b642-026abad804fe None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:15.770757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d9cbb-b760-4129-b642-026abad804fe None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:15.813679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad54d9-8279-4877-9391-6a29d26cf882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:15.815218 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad54d9-8279-4877-9391-6a29d26cf882 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:15.815685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad54d9-8279-4877-9391-6a29d26cf882 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:15.816067 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ad54d9-8279-4877-9391-6a29d26cf882 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:15.831267 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ad54d9-8279-4877-9391-6a29d26cf882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:18:15.836490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-89ad54d9-8279-4877-9391-6a29d26cf882 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:18:15.836490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:15.836490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:15.836490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:15.836490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:15.836490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:15.836490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:15.836490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:15.836490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:15.836490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:15.836490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:15.836490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:15.836490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:18:15.836490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:18:15.836490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:18:15.836490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:18:15.839882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:18:15.839882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:18:15.839882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:18:15.839882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:18:15.839882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:15.839882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:15.839882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:15.839882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:15.839882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:15.839882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:15.839882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:15.839882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:15.839882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:15.839882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:15.839882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:15.839882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:15.840779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:15.840779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:15.840779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:15.840779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:15.840779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:18:15.840779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:18:15.840779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:18:15.840779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:18:15.840779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:18:15.840779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:18:15.840779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:18:15.840779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:15.840779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 605/1222] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:18:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:15.844530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c98a4e-23b4-4e0c-b545-5c31ba06084f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:15.951621 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c98a4e-23b4-4e0c-b545-5c31ba06084f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:15.981079 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:15.995359 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf1b5f2-61e7-4409-b222-660532fb54da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:15.996149 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1b5f2-61e7-4409-b222-660532fb54da None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:15.996545 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1b5f2-61e7-4409-b222-660532fb54da None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:15.996828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf1b5f2-61e7-4409-b222-660532fb54da None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:15.997821 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf1b5f2-61e7-4409-b222-660532fb54da None admin] RBAC: Adding query filter params (name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:15.998238 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf1b5f2-61e7-4409-b222-660532fb54da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:15.999623 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf1b5f2-61e7-4409-b222-660532fb54da None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:16.008962 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 618/1223] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:18:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:16.014979 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad54d9-8279-4877-9391-6a29d26cf882 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:16.063366 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:16.123930 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ad54d9-8279-4877-9391-6a29d26cf882 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:16.167420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e217743-92ca-401b-99ea-8dc557f67eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:16.168094 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e217743-92ca-401b-99ea-8dc557f67eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:16.168556 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e217743-92ca-401b-99ea-8dc557f67eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:16.168831 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e217743-92ca-401b-99ea-8dc557f67eb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:16.182702 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e217743-92ca-401b-99ea-8dc557f67eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:16.185021 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e217743-92ca-401b-99ea-8dc557f67eb8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:16.202544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-2e217743-92ca-401b-99ea-8dc557f67eb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:16.202544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:16.202544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:16.202544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:16.202544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:16.202544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:16.202544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:16.202544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:16.202544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:16.202544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:16.202544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:16.202544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:16.202544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:18:16.202544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:18:16.202544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:18:16.203699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:18:16.203699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:16.203699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:16.203699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:16.203699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:16.203699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:16.203699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:16.203699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:16.203699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:16.203699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:16.203699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:16.203699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:16.203699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:16.203699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:16.203699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:16.203699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:16.203699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:16.204754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:18:16.204754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:18:16.204754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:16.204754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:16.204754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 606/1224] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:18:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:16.209776 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf1b5f2-61e7-4409-b222-660532fb54da None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:16.319365 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf1b5f2-61e7-4409-b222-660532fb54da None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:16.362165 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463e319-c0bf-4191-8ff5-68cc62914b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:16.363083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463e319-c0bf-4191-8ff5-68cc62914b88 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:16.363519 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463e319-c0bf-4191-8ff5-68cc62914b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:16.363979 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463e319-c0bf-4191-8ff5-68cc62914b88 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:16.364918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463e319-c0bf-4191-8ff5-68cc62914b88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:16.365376 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463e319-c0bf-4191-8ff5-68cc62914b88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:16.366960 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463e319-c0bf-4191-8ff5-68cc62914b88 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:16.379706 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 619/1225] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:16.385346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e217743-92ca-401b-99ea-8dc557f67eb8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:16.493308 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:16.500677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e217743-92ca-401b-99ea-8dc557f67eb8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:16.534863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d099f-9ea0-48c9-8db5-713ea4a791e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:16.535515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d099f-9ea0-48c9-8db5-713ea4a791e0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:16.535794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d099f-9ea0-48c9-8db5-713ea4a791e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:16.536105 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d099f-9ea0-48c9-8db5-713ea4a791e0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:16.545133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d099f-9ea0-48c9-8db5-713ea4a791e0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:16.546900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d099f-9ea0-48c9-8db5-713ea4a791e0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:16.560207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-d43d099f-9ea0-48c9-8db5-713ea4a791e0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 22:18:16.560207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:16.560207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:16.560207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:16.560207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:16.560207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:16.560207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:16.560207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:16.560207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:16.560207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:16.560207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:16.560207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:16.560207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:18:16.560207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:18:16.560207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:18:16.560207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:18:16.561478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:16.561478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:16.561478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:18:16.561478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:18:16.561478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:18:16.561478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:18:16.561478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:16.561478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:16.561478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:16.561478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:16.561478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:16.561478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:16.561478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:16.561478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:16.561478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:16.561478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:16.561478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:16.561478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:16.562610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:16.562610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:16.562610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:18:16.562610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:18:16.562610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:18:16.562610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:18:16.562610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:18:16.562610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:18:16.562610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 22:18:16.562610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:16.562610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 607/1226] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:18:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:18:16.568450 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463e319-c0bf-4191-8ff5-68cc62914b88 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:16.617753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:16.674904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463e319-c0bf-4191-8ff5-68cc62914b88 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:16.709265 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c01cb-3db9-4196-973c-5a1e834cd932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:16.709910 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c01cb-3db9-4196-973c-5a1e834cd932 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:16.710190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c01cb-3db9-4196-973c-5a1e834cd932 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:16.710459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0c01cb-3db9-4196-973c-5a1e834cd932 None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:16.711248 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c01cb-3db9-4196-973c-5a1e834cd932 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:16.711555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c01cb-3db9-4196-973c-5a1e834cd932 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:16.712983 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0c01cb-3db9-4196-973c-5a1e834cd932 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:16.731234 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 620/1227] 10.222.0.39 () {56 vars in 1143 bytes} [Thu Jul 28 22:18:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:16.737314 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d099f-9ea0-48c9-8db5-713ea4a791e0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:16.859337 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d099f-9ea0-48c9-8db5-713ea4a791e0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:16.898279 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19942578-99d4-45af-979c-9d9529f54813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:16.898885 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19942578-99d4-45af-979c-9d9529f54813 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:16.899169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19942578-99d4-45af-979c-9d9529f54813 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:16.899453 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19942578-99d4-45af-979c-9d9529f54813 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:16.911308 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19942578-99d4-45af-979c-9d9529f54813 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:16.912891 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19942578-99d4-45af-979c-9d9529f54813 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:16.928193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-19942578-99d4-45af-979c-9d9529f54813 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:16.928193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:16.928193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:16.928193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:16.928193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:16.928193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:16.928193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:16.928193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:16.928193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:16.928193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:16.928193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:16.928193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:16.928193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:18:16.928193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:18:16.928193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:18:16.929329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:18:16.929329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:16.929329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:16.929329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:16.929329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:16.929329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:16.929329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:16.929329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:16.929329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:16.929329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:16.929329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:16.929329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:16.929329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:16.929329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:16.929329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:16.929329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:16.929329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:16.930569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:18:16.930569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:18:16.930569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:16.930569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:16.930569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 608/1228] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:18:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:18:16.935695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c01cb-3db9-4196-973c-5a1e834cd932 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:17.051653 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0c01cb-3db9-4196-973c-5a1e834cd932 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:17.054046 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:17.091025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88fdf39-1f32-4fd7-8214-aeb02354a04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:17.091633 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fdf39-1f32-4fd7-8214-aeb02354a04d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:17.091921 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fdf39-1f32-4fd7-8214-aeb02354a04d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:17.092197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fdf39-1f32-4fd7-8214-aeb02354a04d None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:17.092972 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88fdf39-1f32-4fd7-8214-aeb02354a04d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:17.093190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88fdf39-1f32-4fd7-8214-aeb02354a04d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:17.094601 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88fdf39-1f32-4fd7-8214-aeb02354a04d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:17.106462 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 621/1229] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:17.113060 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19942578-99d4-45af-979c-9d9529f54813 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:17.122683 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:17.232995 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19942578-99d4-45af-979c-9d9529f54813 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:17.271698 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3da8cf-da8b-40eb-b090-b5b086a078b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:17.272506 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3da8cf-da8b-40eb-b090-b5b086a078b7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:17.272824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3da8cf-da8b-40eb-b090-b5b086a078b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:17.273185 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3da8cf-da8b-40eb-b090-b5b086a078b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:17.280826 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3da8cf-da8b-40eb-b090-b5b086a078b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:17.282300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3da8cf-da8b-40eb-b090-b5b086a078b7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:17.294552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-6f3da8cf-da8b-40eb-b090-b5b086a078b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:18:17.294552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:17.294552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:17.294552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:17.294552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:17.294552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:17.294552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:17.294552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:17.294552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:17.294552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:17.294552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:17.294552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:17.294552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:18:17.294552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:18:17.294552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:18:17.295656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:18:17.295656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:17.295656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:17.295656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:17.295656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:17.295656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:17.295656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:17.295656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:17.295656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:17.295656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:17.295656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:17.295656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:17.295656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:17.295656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:17.295656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:17.295656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:17.295656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:17.302320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:18:17.302320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:18:17.302320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:18:17.302320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:18:17.302320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:18:17.302320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:18:17.302320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:18:17.302320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:17.302320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 609/1230] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:18:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:17.304018 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88fdf39-1f32-4fd7-8214-aeb02354a04d None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:17.419999 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88fdf39-1f32-4fd7-8214-aeb02354a04d None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:17.467010 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb90f9-9ade-4be8-8636-262f0740de64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:17.467851 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb90f9-9ade-4be8-8636-262f0740de64 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:17.468294 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb90f9-9ade-4be8-8636-262f0740de64 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:17.468702 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb90f9-9ade-4be8-8636-262f0740de64 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:17.469591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb90f9-9ade-4be8-8636-262f0740de64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:17.469937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb90f9-9ade-4be8-8636-262f0740de64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:17.472196 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb90f9-9ade-4be8-8636-262f0740de64 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:17.485037 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 622/1231] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:18:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:17.491964 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3da8cf-da8b-40eb-b090-b5b086a078b7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:17.568786 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:17.615549 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3da8cf-da8b-40eb-b090-b5b086a078b7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:17.669701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3333ea9-0988-4816-8936-c68e635097d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:17.670967 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3333ea9-0988-4816-8936-c68e635097d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:17.671181 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3333ea9-0988-4816-8936-c68e635097d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:17.671545 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3333ea9-0988-4816-8936-c68e635097d0 None admin] PATH_INFO: `/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/fe003cd7bfbe4b92b1d64d300c3e7d8b/roles/a66ab77a63174592a1e468d2464e8bc4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:17.694666 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3333ea9-0988-4816-8936-c68e635097d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user_id=fe003cd7bfbe4b92b1d64d300c3e7d8b, role_id=a66ab77a63174592a1e468d2464e8bc4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:17.696191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3333ea9-0988-4816-8936-c68e635097d0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:17.712051 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 610/1232] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:18:17 2022] PUT /identity/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/fe003cd7bfbe4b92b1d64d300c3e7d8b/roles/a66ab77a63174592a1e468d2464e8bc4 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:18:19.766552 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6966c93-887e-4c7b-abb7-d9767a2ac913 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:19.767631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6966c93-887e-4c7b-abb7-d9767a2ac913 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:19.767744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6966c93-887e-4c7b-abb7-d9767a2ac913 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:19.769280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 623/1233] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:18:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:18:19.775488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc07f3e-ec56-44a8-9a5e-409a0667c904 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:19.775899 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc07f3e-ec56-44a8-9a5e-409a0667c904 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:19.776105 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc07f3e-ec56-44a8-9a5e-409a0667c904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:19.794976 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6bc07f3e-ec56-44a8-9a5e-409a0667c904 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:18:19.796653 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6bc07f3e-ec56-44a8-9a5e-409a0667c904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:18:19.873199 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 611/1234] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:18:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:18:19.898722 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6966c93-887e-4c7b-abb7-d9767a2ac913 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:19.924629 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:20.011325 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6966c93-887e-4c7b-abb7-d9767a2ac913 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:20.049741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2a7ee-d470-47fe-923f-c34aa9445eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:20.050335 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2a7ee-d470-47fe-923f-c34aa9445eaf None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:20.050618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2a7ee-d470-47fe-923f-c34aa9445eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:20.050968 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f2a7ee-d470-47fe-923f-c34aa9445eaf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:20.062496 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f2a7ee-d470-47fe-923f-c34aa9445eaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:18:20.068616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-c9f2a7ee-d470-47fe-923f-c34aa9445eaf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:18:20.068616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:20.068616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:20.068616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:20.068616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:20.068616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:20.068616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:20.068616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:20.068616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:20.068616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:20.068616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:20.068616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:20.068616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:18:20.068616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:18:20.068616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:18:20.068616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:18:20.069620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:18:20.069620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:18:20.069620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:18:20.069620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:18:20.069620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:20.069620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:20.069620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:20.069620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:20.069620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:20.069620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:20.069620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:20.069620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:20.069620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:20.069620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:20.069620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:20.069620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:20.070503 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:20.070503 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:20.070503 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:20.070503 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:20.070503 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:18:20.070503 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:18:20.070503 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:18:20.070503 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:18:20.070503 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:18:20.070503 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:18:20.070503 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 28 22:18:20.070503 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:18:20.070503 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 624/1235] 10.222.0.39 () {56 vars in 1102 bytes} [Thu Jul 28 22:18:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:20.076699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc07f3e-ec56-44a8-9a5e-409a0667c904 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:20.124246 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:20.180641 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc07f3e-ec56-44a8-9a5e-409a0667c904 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:20.217342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0190ba9-4fc1-4993-9359-cba125575e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:20.218004 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0190ba9-4fc1-4993-9359-cba125575e4b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:20.218295 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0190ba9-4fc1-4993-9359-cba125575e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:20.218546 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0190ba9-4fc1-4993-9359-cba125575e4b None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:20.219265 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0190ba9-4fc1-4993-9359-cba125575e4b None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:20.219567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0190ba9-4fc1-4993-9359-cba125575e4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:20.221540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0190ba9-4fc1-4993-9359-cba125575e4b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:20.230085 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 612/1236] 10.222.0.39 () {56 vars in 1103 bytes} [Thu Jul 28 22:18:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:20.236267 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2a7ee-d470-47fe-923f-c34aa9445eaf None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:20.328417 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:20.347946 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f2a7ee-d470-47fe-923f-c34aa9445eaf None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:20.385256 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec4da8-1a6c-4840-9ecb-6b24f662dda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:20.386182 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec4da8-1a6c-4840-9ecb-6b24f662dda5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:20.386461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec4da8-1a6c-4840-9ecb-6b24f662dda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:20.386731 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec4da8-1a6c-4840-9ecb-6b24f662dda5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:20.399084 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec4da8-1a6c-4840-9ecb-6b24f662dda5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:20.400753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ec4da8-1a6c-4840-9ecb-6b24f662dda5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:20.420041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-b7ec4da8-1a6c-4840-9ecb-6b24f662dda5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:20.420041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:20.420041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:20.420041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:20.420041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:20.420041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:20.420041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:20.420041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:20.420041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:20.420041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:20.420041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:20.420041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:20.420041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:18:20.420041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:18:20.420041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:18:20.421202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:18:20.421202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:20.421202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:20.421202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:20.421202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:20.421202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:20.421202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:20.421202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:20.421202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:20.421202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:20.421202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:20.421202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:20.421202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:20.421202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:20.421202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:20.421202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:20.421202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:20.422433 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:18:20.422433 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:18:20.422433 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:20.422433 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:18:20.422433 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 625/1237] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:20.427913 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0190ba9-4fc1-4993-9359-cba125575e4b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:20.543055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0190ba9-4fc1-4993-9359-cba125575e4b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:20.554935 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:20.584017 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04616fea-01b6-40e6-be0c-2695b150f490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:20.584556 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04616fea-01b6-40e6-be0c-2695b150f490 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:20.584834 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04616fea-01b6-40e6-be0c-2695b150f490 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:20.585106 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04616fea-01b6-40e6-be0c-2695b150f490 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:20.585845 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04616fea-01b6-40e6-be0c-2695b150f490 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:20.586158 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04616fea-01b6-40e6-be0c-2695b150f490 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:20.587523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04616fea-01b6-40e6-be0c-2695b150f490 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:20.600504 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 613/1238] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:20.606484 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec4da8-1a6c-4840-9ecb-6b24f662dda5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:20.715565 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec4da8-1a6c-4840-9ecb-6b24f662dda5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:20.749677 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed47add-ec05-46ac-986f-e9935d056564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:20.750316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed47add-ec05-46ac-986f-e9935d056564 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:20.750597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed47add-ec05-46ac-986f-e9935d056564 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:20.750869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed47add-ec05-46ac-986f-e9935d056564 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:20.754294 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:20.760698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed47add-ec05-46ac-986f-e9935d056564 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:20.762777 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed47add-ec05-46ac-986f-e9935d056564 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:20.776453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-7ed47add-ec05-46ac-986f-e9935d056564 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 22:18:20.776453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:20.776453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:20.776453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:20.776453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:20.776453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:20.776453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:20.776453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:20.776453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:20.776453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:20.776453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:20.776453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:20.776453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:18:20.776453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:18:20.776453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:18:20.776453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:18:20.778106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:20.778106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:20.778106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:18:20.778106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:18:20.778106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:18:20.778106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:18:20.778106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:20.778106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:20.778106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:20.778106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:20.778106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:20.778106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:20.778106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:20.778106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:20.778106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:20.778106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:20.778106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:20.778106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:20.779250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:20.779250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:20.779250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:18:20.779250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:18:20.779250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:18:20.779250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:18:20.779250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:18:20.779250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:18:20.779250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 22:18:20.779250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:18:20.779250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 626/1239] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:18:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:20.785078 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04616fea-01b6-40e6-be0c-2695b150f490 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:20.897846 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04616fea-01b6-40e6-be0c-2695b150f490 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:20.935456 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2549d-319f-404c-8bd8-4ee3c411b605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:20.936107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2549d-319f-404c-8bd8-4ee3c411b605 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:20.936412 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2549d-319f-404c-8bd8-4ee3c411b605 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:20.936829 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2549d-319f-404c-8bd8-4ee3c411b605 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:20.937759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2549d-319f-404c-8bd8-4ee3c411b605 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:20.938058 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2549d-319f-404c-8bd8-4ee3c411b605 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:20.940270 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2549d-319f-404c-8bd8-4ee3c411b605 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:20.962373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 614/1240] 10.222.0.39 () {56 vars in 1143 bytes} [Thu Jul 28 22:18:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:20.969346 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed47add-ec05-46ac-986f-e9935d056564 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:21.080416 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed47add-ec05-46ac-986f-e9935d056564 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:21.121595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92fce6-be31-4e0b-b8c5-74c14d71114a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:21.122338 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92fce6-be31-4e0b-b8c5-74c14d71114a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:21.122700 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92fce6-be31-4e0b-b8c5-74c14d71114a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:21.123103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f92fce6-be31-4e0b-b8c5-74c14d71114a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:21.196942 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:21.205580 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92fce6-be31-4e0b-b8c5-74c14d71114a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:21.207621 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f92fce6-be31-4e0b-b8c5-74c14d71114a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:21.320616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 627/1241] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:18:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 22:18:21.326340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2549d-319f-404c-8bd8-4ee3c411b605 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:21.362243 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:21.448571 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2549d-319f-404c-8bd8-4ee3c411b605 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:21.488614 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be660a1e-d260-4cd9-90ae-c851d8ee7ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:21.489252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be660a1e-d260-4cd9-90ae-c851d8ee7ace None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:21.489557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be660a1e-d260-4cd9-90ae-c851d8ee7ace None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:21.489984 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be660a1e-d260-4cd9-90ae-c851d8ee7ace None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:21.503508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be660a1e-d260-4cd9-90ae-c851d8ee7ace None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:21.505979 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be660a1e-d260-4cd9-90ae-c851d8ee7ace None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:21.525714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-be660a1e-d260-4cd9-90ae-c851d8ee7ace None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:21.525714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:21.525714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:21.525714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:21.525714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:21.525714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:21.525714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:21.525714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:21.525714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:21.525714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:21.525714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:21.525714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:21.525714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:18:21.525714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:18:21.525714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:18:21.528896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:18:21.528896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:21.528896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:21.528896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:21.528896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:21.528896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:21.528896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:21.528896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:21.528896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:21.528896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:21.528896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:21.528896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:21.528896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:21.528896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:21.528896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:21.528896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:21.528896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:21.530718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:18:21.530718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:18:21.530718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:21.530718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:21.530718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 615/1242] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:18:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 28 22:18:21.534119 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92fce6-be31-4e0b-b8c5-74c14d71114a None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:21.654314 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:21.663202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f92fce6-be31-4e0b-b8c5-74c14d71114a None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:21.704696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad986d-4cd4-4e0d-8f3f-fffd58beabd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:21.705373 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad986d-4cd4-4e0d-8f3f-fffd58beabd5 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:21.705690 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad986d-4cd4-4e0d-8f3f-fffd58beabd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:21.705971 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ad986d-4cd4-4e0d-8f3f-fffd58beabd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:21.706676 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad986d-4cd4-4e0d-8f3f-fffd58beabd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:21.706989 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad986d-4cd4-4e0d-8f3f-fffd58beabd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:21.708365 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ad986d-4cd4-4e0d-8f3f-fffd58beabd5 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:21.719848 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 628/1243] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:21.726052 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be660a1e-d260-4cd9-90ae-c851d8ee7ace None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:21.823686 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:21.856351 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be660a1e-d260-4cd9-90ae-c851d8ee7ace None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:21.896502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d24418-d35c-4010-818c-7d1001b48240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:21.897377 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d24418-d35c-4010-818c-7d1001b48240 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:21.897761 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d24418-d35c-4010-818c-7d1001b48240 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:21.898157 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d24418-d35c-4010-818c-7d1001b48240 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:21.905919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d24418-d35c-4010-818c-7d1001b48240 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:21.908297 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d24418-d35c-4010-818c-7d1001b48240 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:21.921557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-70d24418-d35c-4010-818c-7d1001b48240 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:18:21.921557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:21.921557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:21.921557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:21.921557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:21.921557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:21.921557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:21.921557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:21.921557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:21.921557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:21.921557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:21.921557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:21.921557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:18:21.921557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:18:21.921557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:18:21.922707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:18:21.922707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:21.922707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:21.922707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:21.922707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:21.922707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:21.922707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:21.922707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:21.922707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:21.922707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:21.922707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:21.922707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:21.922707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:21.922707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:21.922707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:21.922707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:21.922707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:21.924637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:18:21.924637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:18:21.924637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:18:21.924637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:18:21.924637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:18:21.924637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:18:21.924637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:18:21.924637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:21.924637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 616/1244] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:18:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:21.929275 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad986d-4cd4-4e0d-8f3f-fffd58beabd5 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:22.053427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ad986d-4cd4-4e0d-8f3f-fffd58beabd5 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:22.091079 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd240c2f-43ac-4e12-b3e1-546e4ab51ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:22.091720 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd240c2f-43ac-4e12-b3e1-546e4ab51ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:22.092006 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd240c2f-43ac-4e12-b3e1-546e4ab51ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:22.092289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd240c2f-43ac-4e12-b3e1-546e4ab51ab3 None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:22.093065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd240c2f-43ac-4e12-b3e1-546e4ab51ab3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:22.093379 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd240c2f-43ac-4e12-b3e1-546e4ab51ab3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:22.094894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd240c2f-43ac-4e12-b3e1-546e4ab51ab3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:22.108060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 629/1245] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:18:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:22.114648 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d24418-d35c-4010-818c-7d1001b48240 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:22.241105 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d24418-d35c-4010-818c-7d1001b48240 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:22.279923 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68756344-ae6e-4e7a-a847-0b6a1dc24779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:22.280529 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68756344-ae6e-4e7a-a847-0b6a1dc24779 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:22.280801 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68756344-ae6e-4e7a-a847-0b6a1dc24779 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:22.281070 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68756344-ae6e-4e7a-a847-0b6a1dc24779 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:22.281889 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68756344-ae6e-4e7a-a847-0b6a1dc24779 None admin] RBAC: Adding query filter params (role.id=a66ab77a63174592a1e468d2464e8bc4, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=fe003cd7bfbe4b92b1d64d300c3e7d8b) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:22.282220 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68756344-ae6e-4e7a-a847-0b6a1dc24779 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:22.283638 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68756344-ae6e-4e7a-a847-0b6a1dc24779 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:22.289415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 617/1246] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:18:22 2022] GET /identity/v3/role_assignments?user.id=fe003cd7bfbe4b92b1d64d300c3e7d8b&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=a66ab77a63174592a1e468d2464e8bc4 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:24.389226 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bc215-698d-4835-8730-db85bb67d05a None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:24.389807 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bc215-698d-4835-8730-db85bb67d05a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:24.389807 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bc215-698d-4835-8730-db85bb67d05a None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:24.391196 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 630/1247] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:18:24.396929 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6ed5f-15da-4c96-92c1-99b5e94cd50a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:24.397367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6ed5f-15da-4c96-92c1-99b5e94cd50a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:24.397461 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6ed5f-15da-4c96-92c1-99b5e94cd50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:24.403839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:24.417834 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f0f6ed5f-15da-4c96-92c1-99b5e94cd50a None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:18:24.419140 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f0f6ed5f-15da-4c96-92c1-99b5e94cd50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:18:24.499242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 618/1248] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:18:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:18:24.526053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bc215-698d-4835-8730-db85bb67d05a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:24.548063 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:24.647972 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bc215-698d-4835-8730-db85bb67d05a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:24.687518 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c800a16-8b42-4ef2-ae8e-1c85608b9d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:24.688222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c800a16-8b42-4ef2-ae8e-1c85608b9d36 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:24.688640 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c800a16-8b42-4ef2-ae8e-1c85608b9d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:24.689015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c800a16-8b42-4ef2-ae8e-1c85608b9d36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:24.703266 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c800a16-8b42-4ef2-ae8e-1c85608b9d36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:18:24.709418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-0c800a16-8b42-4ef2-ae8e-1c85608b9d36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:18:24.709418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:24.709418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:24.709418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:24.709418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:24.709418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:24.709418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:24.709418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:24.709418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:24.709418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:24.709418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:24.709418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:24.709418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:18:24.709418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:18:24.709418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:18:24.709418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:18:24.710551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:18:24.710551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:18:24.710551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:18:24.710551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:18:24.710551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:24.710551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:24.710551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:24.710551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:24.710551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:24.710551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:24.710551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:24.710551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:24.710551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:24.710551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:24.710551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:24.710551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:24.711430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:24.711430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:24.711430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:24.711430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:24.711430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:18:24.711430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:18:24.711430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:18:24.711430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:18:24.711430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:18:24.711430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:18:24.711430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:18:24.711430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:18:24.711430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 631/1249] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:18:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:24.717138 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6ed5f-15da-4c96-92c1-99b5e94cd50a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:24.838408 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6ed5f-15da-4c96-92c1-99b5e94cd50a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:24.877253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:24.883410 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6253a6-3c6f-44fa-8ae8-ae86b96d0820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:24.884130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6253a6-3c6f-44fa-8ae8-ae86b96d0820 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:24.884392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6253a6-3c6f-44fa-8ae8-ae86b96d0820 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:24.884691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6253a6-3c6f-44fa-8ae8-ae86b96d0820 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:24.885508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6253a6-3c6f-44fa-8ae8-ae86b96d0820 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:24.886134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6253a6-3c6f-44fa-8ae8-ae86b96d0820 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:24.887646 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6253a6-3c6f-44fa-8ae8-ae86b96d0820 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:24.898105 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 619/1250] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:18:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:24.904150 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c800a16-8b42-4ef2-ae8e-1c85608b9d36 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:24.999887 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:25.028854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c800a16-8b42-4ef2-ae8e-1c85608b9d36 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:25.074667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366a506-8348-4b66-830a-aaed82e05b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:25.075320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366a506-8348-4b66-830a-aaed82e05b14 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:25.075604 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366a506-8348-4b66-830a-aaed82e05b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:25.075878 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366a506-8348-4b66-830a-aaed82e05b14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:25.090480 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366a506-8348-4b66-830a-aaed82e05b14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:25.091950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366a506-8348-4b66-830a-aaed82e05b14 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:25.109220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-c366a506-8348-4b66-830a-aaed82e05b14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:25.109220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:25.109220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:25.109220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:25.109220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:25.109220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:25.109220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:25.109220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:25.109220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:25.109220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:25.109220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:25.109220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:25.109220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:18:25.109220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:18:25.109220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:18:25.110316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:18:25.110316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:25.110316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:25.110316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:25.110316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:25.110316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:25.110316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:25.110316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:25.110316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:25.110316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:25.110316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:25.110316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:25.110316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:25.110316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:25.110316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:25.110316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:25.110316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:25.111736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:18:25.111736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:18:25.111736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:25.111736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:18:25.116218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 632/1251] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:25.117079 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6253a6-3c6f-44fa-8ae8-ae86b96d0820 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:25.229842 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6253a6-3c6f-44fa-8ae8-ae86b96d0820 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:25.269338 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6ba81-fc65-492c-9c39-4d9c7c9033fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:25.270124 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6ba81-fc65-492c-9c39-4d9c7c9033fa None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:25.270531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6ba81-fc65-492c-9c39-4d9c7c9033fa None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:25.270898 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d6ba81-fc65-492c-9c39-4d9c7c9033fa None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:25.271792 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d6ba81-fc65-492c-9c39-4d9c7c9033fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:25.272166 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d6ba81-fc65-492c-9c39-4d9c7c9033fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:25.273832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d6ba81-fc65-492c-9c39-4d9c7c9033fa None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:25.288651 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 620/1252] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:18:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:25.294867 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366a506-8348-4b66-830a-aaed82e05b14 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:25.413596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366a506-8348-4b66-830a-aaed82e05b14 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:25.441565 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:25.450875 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62dea4-e2dc-400e-8382-2244a69231d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:25.451532 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62dea4-e2dc-400e-8382-2244a69231d1 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:25.451817 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62dea4-e2dc-400e-8382-2244a69231d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:25.452097 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed62dea4-e2dc-400e-8382-2244a69231d1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:25.461674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62dea4-e2dc-400e-8382-2244a69231d1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:25.463131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed62dea4-e2dc-400e-8382-2244a69231d1 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:25.476136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-ed62dea4-e2dc-400e-8382-2244a69231d1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 22:18:25.476136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:25.476136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:25.476136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:25.476136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:25.476136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:25.476136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:25.476136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:25.476136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:25.476136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:25.476136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:25.476136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:25.476136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:18:25.476136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:18:25.476136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:18:25.476136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:18:25.477284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:25.477284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:25.477284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:18:25.477284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:18:25.477284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:18:25.477284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:18:25.477284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:25.477284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:25.477284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:25.477284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:25.477284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:25.477284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:25.477284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:25.477284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:25.477284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:25.477284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:25.477284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:25.477284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:25.478292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:25.478292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:25.478292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:18:25.478292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:18:25.478292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:18:25.478292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:18:25.478292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:18:25.478292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:18:25.478292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 22:18:25.478292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:18:25.478292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 633/1253] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:18:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:25.483830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6ba81-fc65-492c-9c39-4d9c7c9033fa None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:25.546709 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:25.597660 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d6ba81-fc65-492c-9c39-4d9c7c9033fa None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:25.634634 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ef00f-b89e-47e0-a5d1-51bd49dc4c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:25.635274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ef00f-b89e-47e0-a5d1-51bd49dc4c50 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:25.635557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ef00f-b89e-47e0-a5d1-51bd49dc4c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:25.636106 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9ef00f-b89e-47e0-a5d1-51bd49dc4c50 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:25.636927 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ef00f-b89e-47e0-a5d1-51bd49dc4c50 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:25.637242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ef00f-b89e-47e0-a5d1-51bd49dc4c50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:25.638920 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9ef00f-b89e-47e0-a5d1-51bd49dc4c50 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:25.661514 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 621/1254] 10.222.0.39 () {56 vars in 1143 bytes} [Thu Jul 28 22:18:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:25.668191 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62dea4-e2dc-400e-8382-2244a69231d1 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:25.779036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed62dea4-e2dc-400e-8382-2244a69231d1 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:25.819406 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad81dc-348c-4c9d-a9d5-5c3209af2a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:25.820185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad81dc-348c-4c9d-a9d5-5c3209af2a6c None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:25.820719 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad81dc-348c-4c9d-a9d5-5c3209af2a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:25.821362 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad81dc-348c-4c9d-a9d5-5c3209af2a6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:25.888162 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:25.901994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad81dc-348c-4c9d-a9d5-5c3209af2a6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:25.903598 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad81dc-348c-4c9d-a9d5-5c3209af2a6c None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:26.011572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 634/1255] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:18:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 22:18:26.018208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ef00f-b89e-47e0-a5d1-51bd49dc4c50 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:26.130015 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9ef00f-b89e-47e0-a5d1-51bd49dc4c50 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:26.148451 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:26.174509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea6070-2ed3-4fb8-bbd9-a6f209918517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:26.175176 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea6070-2ed3-4fb8-bbd9-a6f209918517 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:26.175455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea6070-2ed3-4fb8-bbd9-a6f209918517 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:26.175721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ea6070-2ed3-4fb8-bbd9-a6f209918517 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:26.188236 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea6070-2ed3-4fb8-bbd9-a6f209918517 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:26.189979 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ea6070-2ed3-4fb8-bbd9-a6f209918517 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:26.208778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-b9ea6070-2ed3-4fb8-bbd9-a6f209918517 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:26.208778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:26.208778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:26.208778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:26.208778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:26.208778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:26.208778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:26.208778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:26.208778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:26.208778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:26.208778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:26.208778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:26.208778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:18:26.208778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:18:26.208778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:18:26.209782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:18:26.209782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:26.209782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:26.209782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:26.209782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:26.209782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:26.209782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:26.209782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:26.209782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:26.209782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:26.209782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:26.209782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:26.209782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:26.209782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:26.209782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:26.209782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:26.209782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:26.214033 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:18:26.214033 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:18:26.214033 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:26.214033 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:26.214033 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 622/1256] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:18:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:26.216578 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad81dc-348c-4c9d-a9d5-5c3209af2a6c None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:26.309067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:26.322632 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad81dc-348c-4c9d-a9d5-5c3209af2a6c None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:26.367135 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a954cc-59ce-4ca6-891d-1bcef0d3e8c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:26.367758 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a954cc-59ce-4ca6-891d-1bcef0d3e8c3 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:26.368034 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a954cc-59ce-4ca6-891d-1bcef0d3e8c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:26.368306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a954cc-59ce-4ca6-891d-1bcef0d3e8c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:26.368995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a954cc-59ce-4ca6-891d-1bcef0d3e8c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:26.369294 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a954cc-59ce-4ca6-891d-1bcef0d3e8c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:26.370690 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a954cc-59ce-4ca6-891d-1bcef0d3e8c3 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:26.383007 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 635/1257] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:18:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:26.389022 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea6070-2ed3-4fb8-bbd9-a6f209918517 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:26.505393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ea6070-2ed3-4fb8-bbd9-a6f209918517 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:26.541759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42317cc2-aa72-4c5d-b5b2-0c8a61046e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:26.542438 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42317cc2-aa72-4c5d-b5b2-0c8a61046e34 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:26.542892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42317cc2-aa72-4c5d-b5b2-0c8a61046e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:26.543203 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42317cc2-aa72-4c5d-b5b2-0c8a61046e34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:26.552650 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42317cc2-aa72-4c5d-b5b2-0c8a61046e34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:26.554643 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42317cc2-aa72-4c5d-b5b2-0c8a61046e34 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:26.569480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-42317cc2-aa72-4c5d-b5b2-0c8a61046e34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:18:26.569480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:26.569480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:26.569480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:26.569480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:26.569480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:26.569480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:26.569480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:26.569480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:26.569480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:26.569480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:26.569480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:26.569480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:18:26.569480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:18:26.569480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:18:26.570625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:18:26.570625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:26.570625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:26.570625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:26.570625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:26.570625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:26.570625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:26.570625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:26.570625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:26.570625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:26.570625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:26.570625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:26.570625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:26.570625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:26.570625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:26.570625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:26.570625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:26.571757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:18:26.571757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:18:26.571757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:18:26.571757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:18:26.571757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:18:26.571757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:18:26.571757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:18:26.571757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:26.571757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 623/1258] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:18:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:26.576692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a954cc-59ce-4ca6-891d-1bcef0d3e8c3 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:26.692103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a954cc-59ce-4ca6-891d-1bcef0d3e8c3 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:26.726396 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba9c4a-df30-4683-9d6e-4a782a2f6a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:26.727012 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba9c4a-df30-4683-9d6e-4a782a2f6a3a None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:26.727296 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba9c4a-df30-4683-9d6e-4a782a2f6a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:26.727566 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba9c4a-df30-4683-9d6e-4a782a2f6a3a None admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:26.728313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba9c4a-df30-4683-9d6e-4a782a2f6a3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:26.728613 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba9c4a-df30-4683-9d6e-4a782a2f6a3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:26.730089 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba9c4a-df30-4683-9d6e-4a782a2f6a3a None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:26.743019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 636/1259] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:18:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:26.749384 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42317cc2-aa72-4c5d-b5b2-0c8a61046e34 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:26.776745 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:26.871319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42317cc2-aa72-4c5d-b5b2-0c8a61046e34 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:26.922125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caadd40e-b5a2-4f57-a422-a165c460e4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:26.923354 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caadd40e-b5a2-4f57-a422-a165c460e4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:26.923629 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caadd40e-b5a2-4f57-a422-a165c460e4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:26.923901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caadd40e-b5a2-4f57-a422-a165c460e4a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:26.924679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caadd40e-b5a2-4f57-a422-a165c460e4a0 None admin] RBAC: Adding query filter params (role.id=aded7bd43e974686b3c23bd9d4255b24, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=fe003cd7bfbe4b92b1d64d300c3e7d8b) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:26.924991 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caadd40e-b5a2-4f57-a422-a165c460e4a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:26.926462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caadd40e-b5a2-4f57-a422-a165c460e4a0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:26.932420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 624/1260] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:18:26 2022] GET /identity/v3/role_assignments?user.id=fe003cd7bfbe4b92b1d64d300c3e7d8b&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=aded7bd43e974686b3c23bd9d4255b24 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:28.974950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a0deb-b5aa-43eb-98ba-6f86e4f83404 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:28.975479 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a0deb-b5aa-43eb-98ba-6f86e4f83404 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:28.975479 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4a0deb-b5aa-43eb-98ba-6f86e4f83404 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:28.976866 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 637/1261] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:18:28.981915 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5cbf2-7c91-474b-abfb-fdf290ad667d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:28.982285 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5cbf2-7c91-474b-abfb-fdf290ad667d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:28.982468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5cbf2-7c91-474b-abfb-fdf290ad667d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:29.000208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-49e5cbf2-7c91-474b-abfb-fdf290ad667d None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:18:29.001403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-49e5cbf2-7c91-474b-abfb-fdf290ad667d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:18:29.070974 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 625/1262] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:18:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:18:29.093199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a0deb-b5aa-43eb-98ba-6f86e4f83404 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:29.104664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:29.194724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4a0deb-b5aa-43eb-98ba-6f86e4f83404 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:29.227363 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b031e-8a4b-46bd-bef8-7fdef150100f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:29.227935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b031e-8a4b-46bd-bef8-7fdef150100f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:29.228220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b031e-8a4b-46bd-bef8-7fdef150100f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:29.228497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b031e-8a4b-46bd-bef8-7fdef150100f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:29.238324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b031e-8a4b-46bd-bef8-7fdef150100f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:18:29.243277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-506b031e-8a4b-46bd-bef8-7fdef150100f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:18:29.243277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:29.243277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:29.243277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:29.243277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:29.243277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:29.243277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:29.243277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:29.243277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:29.243277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:29.243277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:29.243277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:29.243277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:18:29.243277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:18:29.243277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:18:29.243277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:18:29.244368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:18:29.244368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:18:29.244368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:18:29.244368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:18:29.244368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:29.244368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:29.244368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:29.244368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:29.244368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:29.244368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:29.244368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:29.244368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:29.244368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:29.244368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:29.244368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:29.244368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:29.245427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:29.245427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:29.245427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:29.245427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:29.245427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:18:29.245427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:18:29.245427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:18:29.245427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:18:29.245427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:18:29.245427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:18:29.245427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:18:29.245427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:18:29.245427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 638/1263] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:18:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:29.250375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5cbf2-7c91-474b-abfb-fdf290ad667d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:29.270049 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:29.349887 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5cbf2-7c91-474b-abfb-fdf290ad667d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:29.383128 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb38b7-f2a9-4326-9183-78e2e6ae542b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:29.383743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb38b7-f2a9-4326-9183-78e2e6ae542b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:29.384026 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb38b7-f2a9-4326-9183-78e2e6ae542b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:29.384300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb38b7-f2a9-4326-9183-78e2e6ae542b None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:29.385019 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb38b7-f2a9-4326-9183-78e2e6ae542b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:29.385324 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb38b7-f2a9-4326-9183-78e2e6ae542b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:29.386990 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb38b7-f2a9-4326-9183-78e2e6ae542b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:29.394876 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 626/1264] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:18:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:29.400569 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b031e-8a4b-46bd-bef8-7fdef150100f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:29.465978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:29.499246 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b031e-8a4b-46bd-bef8-7fdef150100f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:29.531100 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db9710-54c6-4cdb-b6c0-f4184f55b4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:29.532275 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db9710-54c6-4cdb-b6c0-f4184f55b4ad None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:29.532802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db9710-54c6-4cdb-b6c0-f4184f55b4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:29.532802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6db9710-54c6-4cdb-b6c0-f4184f55b4ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:29.542355 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db9710-54c6-4cdb-b6c0-f4184f55b4ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:29.543787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6db9710-54c6-4cdb-b6c0-f4184f55b4ad None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:29.558639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-a6db9710-54c6-4cdb-b6c0-f4184f55b4ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:29.558639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:29.558639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:29.558639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:29.558639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:29.558639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:29.558639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:29.558639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:29.558639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:29.558639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:29.558639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:29.558639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:29.558639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:18:29.558639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:18:29.558639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:18:29.559536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:18:29.559536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:29.559536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:29.559536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:29.559536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:29.559536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:29.559536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:29.559536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:29.559536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:29.559536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:29.559536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:29.559536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:29.559536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:29.559536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:29.559536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:29.559536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:29.559536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:29.560745 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:18:29.560745 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:18:29.560745 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:29.560745 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:18:29.560745 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 639/1265] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:29.565812 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb38b7-f2a9-4326-9183-78e2e6ae542b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:29.647173 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:29.666753 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb38b7-f2a9-4326-9183-78e2e6ae542b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:29.699269 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c87b6-3fd9-4876-9b01-2e70c94464da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:29.699870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c87b6-3fd9-4876-9b01-2e70c94464da None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:29.700146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c87b6-3fd9-4876-9b01-2e70c94464da None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:29.700414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7c87b6-3fd9-4876-9b01-2e70c94464da None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:29.701091 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c87b6-3fd9-4876-9b01-2e70c94464da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:29.701394 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c87b6-3fd9-4876-9b01-2e70c94464da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:29.702768 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7c87b6-3fd9-4876-9b01-2e70c94464da None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:29.714543 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 627/1266] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:18:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:29.720711 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db9710-54c6-4cdb-b6c0-f4184f55b4ad None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:29.825935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6db9710-54c6-4cdb-b6c0-f4184f55b4ad None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:29.851506 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:29.868500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dae0e8-0817-4d8c-8427-04134a0f8804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:29.869512 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dae0e8-0817-4d8c-8427-04134a0f8804 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:29.869892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dae0e8-0817-4d8c-8427-04134a0f8804 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:29.870203 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dae0e8-0817-4d8c-8427-04134a0f8804 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:29.878499 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dae0e8-0817-4d8c-8427-04134a0f8804 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:29.879979 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dae0e8-0817-4d8c-8427-04134a0f8804 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:29.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-97dae0e8-0817-4d8c-8427-04134a0f8804 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 22:18:29.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:29.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:29.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:29.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:29.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:29.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:29.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:29.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:29.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:29.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:29.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:29.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:18:29.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:18:29.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:18:29.892571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:18:29.893749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:29.893749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:29.893749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:18:29.893749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:18:29.893749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:18:29.893749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:18:29.893749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:29.893749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:29.893749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:29.893749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:29.893749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:29.893749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:29.893749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:29.893749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:29.893749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:29.893749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:29.893749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:29.893749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:29.895103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:29.895103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:29.895103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:18:29.895103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:18:29.895103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:18:29.895103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:18:29.895103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:18:29.895103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:18:29.895103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 22:18:29.895103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:18:29.895103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 640/1267] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:18:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:29.899823 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c87b6-3fd9-4876-9b01-2e70c94464da None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:30.003810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7c87b6-3fd9-4876-9b01-2e70c94464da None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:30.037145 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9742f70-af43-4ede-b85d-fefab86fcf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:30.037778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9742f70-af43-4ede-b85d-fefab86fcf54 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:30.038062 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9742f70-af43-4ede-b85d-fefab86fcf54 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:30.038334 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9742f70-af43-4ede-b85d-fefab86fcf54 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:30.039125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9742f70-af43-4ede-b85d-fefab86fcf54 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:30.039431 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9742f70-af43-4ede-b85d-fefab86fcf54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:30.040859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9742f70-af43-4ede-b85d-fefab86fcf54 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:30.049416 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:30.057847 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 628/1268] 10.222.0.39 () {56 vars in 1143 bytes} [Thu Jul 28 22:18:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:30.064206 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dae0e8-0817-4d8c-8427-04134a0f8804 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:30.162223 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dae0e8-0817-4d8c-8427-04134a0f8804 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:30.195021 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d83425-f329-46cb-a87f-3d1d70fd4829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:30.195651 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d83425-f329-46cb-a87f-3d1d70fd4829 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:30.195937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d83425-f329-46cb-a87f-3d1d70fd4829 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:30.196216 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d83425-f329-46cb-a87f-3d1d70fd4829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:30.212452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d83425-f329-46cb-a87f-3d1d70fd4829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:30.213944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d83425-f329-46cb-a87f-3d1d70fd4829 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:30.228872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-27d83425-f329-46cb-a87f-3d1d70fd4829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:30.228872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:30.228872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:30.228872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:30.228872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:30.228872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:30.228872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:30.228872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:30.228872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:30.228872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:30.228872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:30.228872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:30.228872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:18:30.228872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:18:30.228872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:18:30.230067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:18:30.230067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:30.230067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:30.230067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:30.230067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:30.230067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:30.230067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:30.230067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:30.230067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:30.230067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:30.230067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:30.230067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:30.230067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:30.230067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:30.230067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:30.230067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:30.230067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:30.231038 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:18:30.231038 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:18:30.231038 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:30.231038 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:18:30.231038 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 641/1269] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:18:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:30.235840 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9742f70-af43-4ede-b85d-fefab86fcf54 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:30.333798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9742f70-af43-4ede-b85d-fefab86fcf54 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:30.366637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783bf9e8-b579-4b45-a192-20a5bc42b48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:30.367244 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bf9e8-b579-4b45-a192-20a5bc42b48a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:30.367528 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bf9e8-b579-4b45-a192-20a5bc42b48a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:30.367804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783bf9e8-b579-4b45-a192-20a5bc42b48a None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:30.368491 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783bf9e8-b579-4b45-a192-20a5bc42b48a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:30.368801 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783bf9e8-b579-4b45-a192-20a5bc42b48a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:30.370197 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783bf9e8-b579-4b45-a192-20a5bc42b48a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:30.381730 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 629/1270] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:30.387718 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d83425-f329-46cb-a87f-3d1d70fd4829 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:30.419197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:30.487029 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d83425-f329-46cb-a87f-3d1d70fd4829 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:30.520394 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0e4ed-9154-40c5-b482-1b8d8df85d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:30.521023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0e4ed-9154-40c5-b482-1b8d8df85d85 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:30.521308 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0e4ed-9154-40c5-b482-1b8d8df85d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:30.521626 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca0e4ed-9154-40c5-b482-1b8d8df85d85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:30.528056 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0e4ed-9154-40c5-b482-1b8d8df85d85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:30.529534 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca0e4ed-9154-40c5-b482-1b8d8df85d85 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:30.541179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-4ca0e4ed-9154-40c5-b482-1b8d8df85d85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:18:30.541179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:30.541179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:30.541179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:30.541179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:30.541179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:30.541179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:30.541179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:30.541179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:30.541179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:30.541179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:30.541179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:30.541179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:18:30.541179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:18:30.541179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:18:30.542306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:18:30.542306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:30.542306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:30.542306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:30.542306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:30.542306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:30.542306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:30.542306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:30.542306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:30.542306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:30.542306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:30.542306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:30.542306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:30.542306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:30.542306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:30.542306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:30.542306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:30.543544 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:18:30.543544 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:18:30.543544 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:18:30.543544 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:18:30.543544 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:18:30.543544 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:18:30.543544 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:18:30.543544 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:18:30.543544 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 642/1271] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:18:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:30.548818 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783bf9e8-b579-4b45-a192-20a5bc42b48a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:30.605815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:30.646633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783bf9e8-b579-4b45-a192-20a5bc42b48a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:30.679006 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd12c55-d08e-4553-a744-644833298381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:30.679547 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd12c55-d08e-4553-a744-644833298381 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:30.679825 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd12c55-d08e-4553-a744-644833298381 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:30.680096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd12c55-d08e-4553-a744-644833298381 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:30.680836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd12c55-d08e-4553-a744-644833298381 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:30.681141 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd12c55-d08e-4553-a744-644833298381 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:30.682714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd12c55-d08e-4553-a744-644833298381 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:30.694901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 630/1272] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:18:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:30.700920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0e4ed-9154-40c5-b482-1b8d8df85d85 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:30.790543 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:30.801449 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca0e4ed-9154-40c5-b482-1b8d8df85d85 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:30.839714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb2cf8-5379-4b01-8f5e-aac09a904a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:30.840367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb2cf8-5379-4b01-8f5e-aac09a904a29 None admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:30.840645 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb2cf8-5379-4b01-8f5e-aac09a904a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:30.840914 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb2cf8-5379-4b01-8f5e-aac09a904a29 None admin] PATH_INFO: `/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/fe003cd7bfbe4b92b1d64d300c3e7d8b/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:30.842840 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb2cf8-5379-4b01-8f5e-aac09a904a29 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user_id=fe003cd7bfbe4b92b1d64d300c3e7d8b, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:30.844899 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb2cf8-5379-4b01-8f5e-aac09a904a29 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:30.858059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 643/1273] 10.222.0.39 () {58 vars in 1439 bytes} [Thu Jul 28 22:18:30 2022] PUT /identity/v3/projects/b9fb2a1c6ad9453d9daf18d42a6f24ec/users/fe003cd7bfbe4b92b1d64d300c3e7d8b/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:18:32.840263 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978fc8c-e954-49f8-b845-49274d207543 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:32.840821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978fc8c-e954-49f8-b845-49274d207543 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:32.840821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978fc8c-e954-49f8-b845-49274d207543 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:32.842208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 631/1274] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:18:32.847493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc56745-8ad7-42cc-a910-679978898a43 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:32.847892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc56745-8ad7-42cc-a910-679978898a43 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:32.848045 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc56745-8ad7-42cc-a910-679978898a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:32.866115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-2cc56745-8ad7-42cc-a910-679978898a43 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:18:32.867293 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2cc56745-8ad7-42cc-a910-679978898a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:18:32.933171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 644/1275] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:18:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:18:32.956383 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978fc8c-e954-49f8-b845-49274d207543 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:33.055378 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978fc8c-e954-49f8-b845-49274d207543 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:33.071802 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:33.094661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c9071-c5b7-47f5-8f93-b7ee25737bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:33.095469 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c9071-c5b7-47f5-8f93-b7ee25737bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:33.095736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c9071-c5b7-47f5-8f93-b7ee25737bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:33.096005 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c9071-c5b7-47f5-8f93-b7ee25737bf6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:33.105872 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c9071-c5b7-47f5-8f93-b7ee25737bf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 28 22:18:33.111548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-899c9071-c5b7-47f5-8f93-b7ee25737bf6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:18:33.111548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:33.111548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:33.111548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:33.111548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:33.111548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:33.111548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:33.111548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:33.111548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:33.111548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:33.111548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:33.111548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:33.111548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 28 22:18:33.111548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 28 22:18:33.111548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 28 22:18:33.111548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 28 22:18:33.112668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 28 22:18:33.112668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 28 22:18:33.112668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 28 22:18:33.112668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 28 22:18:33.112668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:33.112668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:33.112668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:33.112668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:33.112668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:33.112668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:33.112668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:33.112668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:33.112668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:33.112668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:33.112668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:33.112668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:33.113908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:33.113908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:33.113908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:33.113908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:33.113908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 28 22:18:33.113908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 28 22:18:33.113908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 28 22:18:33.113908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 28 22:18:33.113908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 28 22:18:33.113908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 28 22:18:33.113908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 28 22:18:33.113908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:33.113908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 632/1276] 10.222.0.39 () {56 vars in 1096 bytes} [Thu Jul 28 22:18:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:33.118667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc56745-8ad7-42cc-a910-679978898a43 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:33.197491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:33.215544 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc56745-8ad7-42cc-a910-679978898a43 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:33.248938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2719d5-4a12-4ec1-96b7-c8694a91eab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:33.249564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2719d5-4a12-4ec1-96b7-c8694a91eab6 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:33.249852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2719d5-4a12-4ec1-96b7-c8694a91eab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:33.250127 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2719d5-4a12-4ec1-96b7-c8694a91eab6 None admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:33.251011 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2719d5-4a12-4ec1-96b7-c8694a91eab6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:33.251320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2719d5-4a12-4ec1-96b7-c8694a91eab6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:33.252687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2719d5-4a12-4ec1-96b7-c8694a91eab6 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:33.260704 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 645/1277] 10.222.0.39 () {56 vars in 1099 bytes} [Thu Jul 28 22:18:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:33.267043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c9071-c5b7-47f5-8f93-b7ee25737bf6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:33.366742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c9071-c5b7-47f5-8f93-b7ee25737bf6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:33.399360 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e2dbc-6490-4957-a0ca-151d0437459d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:33.399980 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e2dbc-6490-4957-a0ca-151d0437459d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:33.400257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e2dbc-6490-4957-a0ca-151d0437459d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:33.400522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e2dbc-6490-4957-a0ca-151d0437459d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:33.411079 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e2dbc-6490-4957-a0ca-151d0437459d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:33.412500 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e2dbc-6490-4957-a0ca-151d0437459d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:33.426238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c75e2dbc-6490-4957-a0ca-151d0437459d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:33.426238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:33.426238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:33.426238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:33.426238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:33.426238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:33.426238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:33.426238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:33.426238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:33.426238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:33.426238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:33.426238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:33.426238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:18:33.426238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:18:33.426238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:18:33.427304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:18:33.427304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:33.427304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:33.427304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:33.427304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:33.427304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:33.427304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:33.427304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:33.427304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:33.427304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:33.427304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:33.427304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:33.427304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:33.427304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:33.427304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:33.427304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:33.427304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:33.428392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:18:33.428392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:18:33.428392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:33.428392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:33.428392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 633/1278] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:33.433172 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2719d5-4a12-4ec1-96b7-c8694a91eab6 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:33.531195 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2719d5-4a12-4ec1-96b7-c8694a91eab6 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:33.562986 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2c816-4996-48aa-8bce-695b3d7c14a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:33.563587 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2c816-4996-48aa-8bce-695b3d7c14a4 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:33.563862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2c816-4996-48aa-8bce-695b3d7c14a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:33.564133 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd2c816-4996-48aa-8bce-695b3d7c14a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:33.564820 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2c816-4996-48aa-8bce-695b3d7c14a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:33.565120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2c816-4996-48aa-8bce-695b3d7c14a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:33.566508 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd2c816-4996-48aa-8bce-695b3d7c14a4 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:33.577662 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 646/1279] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:33.583673 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e2dbc-6490-4957-a0ca-151d0437459d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:33.613850 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:33.680785 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e2dbc-6490-4957-a0ca-151d0437459d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:33.713241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172d92d-ff93-4d61-a83f-2d1d8040bbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:33.713880 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172d92d-ff93-4d61-a83f-2d1d8040bbec None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:33.714156 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172d92d-ff93-4d61-a83f-2d1d8040bbec None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:33.714423 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172d92d-ff93-4d61-a83f-2d1d8040bbec None admin] PATH_INFO: `/v3/users/placement` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:33.721884 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172d92d-ff93-4d61-a83f-2d1d8040bbec None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:33.723298 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172d92d-ff93-4d61-a83f-2d1d8040bbec None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:33.734261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-e172d92d-ff93-4d61-a83f-2d1d8040bbec None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 28 22:18:33.734261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:33.734261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:33.734261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:33.734261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:33.734261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:33.734261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:33.734261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:33.734261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:33.734261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:33.734261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:33.734261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:33.734261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 28 22:18:33.734261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 28 22:18:33.734261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 28 22:18:33.734261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 28 22:18:33.735434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:33.735434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:33.735434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 28 22:18:33.735434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:18:33.735434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 28 22:18:33.735434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 28 22:18:33.735434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:33.735434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:33.735434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:33.735434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:33.735434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:33.735434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:33.735434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:33.735434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:33.735434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:33.735434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:33.735434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:33.735434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:33.736508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:33.736508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:33.736508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 28 22:18:33.736508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 28 22:18:33.736508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 28 22:18:33.736508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 28 22:18:33.736508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 28 22:18:33.736508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 28 22:18:33.736508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 28 22:18:33.736508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:33.736508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 634/1280] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:18:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:33.741199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2c816-4996-48aa-8bce-695b3d7c14a4 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:33.744945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:33.843706 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd2c816-4996-48aa-8bce-695b3d7c14a4 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:33.876191 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fbad8-422a-412f-a7f5-f3c43c57b49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:33.876772 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fbad8-422a-412f-a7f5-f3c43c57b49f None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:33.877052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fbad8-422a-412f-a7f5-f3c43c57b49f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:33.877319 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6fbad8-422a-412f-a7f5-f3c43c57b49f None admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:33.878129 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fbad8-422a-412f-a7f5-f3c43c57b49f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:33.878433 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fbad8-422a-412f-a7f5-f3c43c57b49f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:33.879846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6fbad8-422a-412f-a7f5-f3c43c57b49f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:33.896971 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 647/1281] 10.222.0.39 () {56 vars in 1143 bytes} [Thu Jul 28 22:18:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:33.902725 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172d92d-ff93-4d61-a83f-2d1d8040bbec None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:33.989644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:33.999063 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172d92d-ff93-4d61-a83f-2d1d8040bbec None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:34.032366 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1718a-fee9-4a79-9475-e33852ac1f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:34.033093 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1718a-fee9-4a79-9475-e33852ac1f45 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:34.033373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1718a-fee9-4a79-9475-e33852ac1f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:34.033676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae1718a-fee9-4a79-9475-e33852ac1f45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:34.100355 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1718a-fee9-4a79-9475-e33852ac1f45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:34.101892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae1718a-fee9-4a79-9475-e33852ac1f45 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:34.188619 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:34.198864 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 635/1282] 10.222.0.39 () {58 vars in 1282 bytes} [Thu Jul 28 22:18:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 28 22:18:34.205147 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fbad8-422a-412f-a7f5-f3c43c57b49f None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:34.264685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:34.302704 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6fbad8-422a-412f-a7f5-f3c43c57b49f None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:34.335795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cdf91-2cd2-40a6-b044-5f900eef2a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:34.336424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cdf91-2cd2-40a6-b044-5f900eef2a46 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:34.336706 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cdf91-2cd2-40a6-b044-5f900eef2a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:34.337000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272cdf91-2cd2-40a6-b044-5f900eef2a46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:34.347312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272cdf91-2cd2-40a6-b044-5f900eef2a46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:34.348766 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272cdf91-2cd2-40a6-b044-5f900eef2a46 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:34.363181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-272cdf91-2cd2-40a6-b044-5f900eef2a46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:34.363181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:34.363181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:34.363181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:34.363181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:34.363181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:34.363181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:34.363181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:34.363181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:34.363181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:34.363181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:34.363181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:34.363181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 28 22:18:34.363181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 28 22:18:34.363181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 28 22:18:34.364258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 28 22:18:34.364258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:34.364258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:34.364258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:34.364258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:34.364258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:34.364258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:34.364258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:34.364258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:34.364258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:34.364258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:34.364258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:34.364258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:34.364258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:34.364258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:34.364258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:34.364258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:34.365436 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 28 22:18:34.365436 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 28 22:18:34.365436 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 28 22:18:34.365436 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:18:34.368891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 648/1283] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:18:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:34.370070 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1718a-fee9-4a79-9475-e33852ac1f45 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:34.465494 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae1718a-fee9-4a79-9475-e33852ac1f45 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:34.497984 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f2380-d071-4510-815d-9690e14eb561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:34.498577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f2380-d071-4510-815d-9690e14eb561 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:34.498856 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f2380-d071-4510-815d-9690e14eb561 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:34.499122 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f2380-d071-4510-815d-9690e14eb561 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:34.499799 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f2380-d071-4510-815d-9690e14eb561 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:34.500096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f2380-d071-4510-815d-9690e14eb561 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:34.501977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223f2380-d071-4510-815d-9690e14eb561 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:34.513027 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 636/1284] 10.222.0.39 () {56 vars in 1109 bytes} [Thu Jul 28 22:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:34.519266 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cdf91-2cd2-40a6-b044-5f900eef2a46 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:34.617006 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272cdf91-2cd2-40a6-b044-5f900eef2a46 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:34.630567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:34.651035 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804720e-fe8d-4eab-8a26-a9ab1bda17b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:34.651660 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804720e-fe8d-4eab-8a26-a9ab1bda17b7 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:34.651946 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804720e-fe8d-4eab-8a26-a9ab1bda17b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:34.652224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f804720e-fe8d-4eab-8a26-a9ab1bda17b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:34.658727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f804720e-fe8d-4eab-8a26-a9ab1bda17b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:34.660177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f804720e-fe8d-4eab-8a26-a9ab1bda17b7 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:34.671324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application [None req-f804720e-fe8d-4eab-8a26-a9ab1bda17b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:18:34.671324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:34.671324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:34.671324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:34.671324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:34.671324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:34.671324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:34.671324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:34.671324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:34.671324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:34.671324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:34.671324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:34.671324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 28 22:18:34.671324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 28 22:18:34.671324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 28 22:18:34.672594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 28 22:18:34.672594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:34.672594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:34.672594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:34.672594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:34.672594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:34.672594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:34.672594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:34.672594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:34.672594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:34.672594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:34.672594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:34.672594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:34.672594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:34.672594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:34.672594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:34.672594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:34.673809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 28 22:18:34.673809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 28 22:18:34.673809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 28 22:18:34.673809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 28 22:18:34.673809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 28 22:18:34.673809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 28 22:18:34.673809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 28 22:18:34.673809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: ERROR keystone.server.flask.application Jul 28 22:18:34.673809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 649/1285] 10.222.0.39 () {56 vars in 1111 bytes} [Thu Jul 28 22:18:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:34.678337 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f2380-d071-4510-815d-9690e14eb561 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:34.727150 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:34.776690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f2380-d071-4510-815d-9690e14eb561 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:34.813776 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe405edd-20f5-4904-b9f8-434ffc49c462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:34.814519 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe405edd-20f5-4904-b9f8-434ffc49c462 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:34.814787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe405edd-20f5-4904-b9f8-434ffc49c462 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:34.815081 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe405edd-20f5-4904-b9f8-434ffc49c462 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:34.816027 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe405edd-20f5-4904-b9f8-434ffc49c462 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:34.816376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe405edd-20f5-4904-b9f8-434ffc49c462 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:34.818154 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe405edd-20f5-4904-b9f8-434ffc49c462 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:34.831219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 637/1286] 10.222.0.39 () {56 vars in 1148 bytes} [Thu Jul 28 22:18:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:34.837319 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804720e-fe8d-4eab-8a26-a9ab1bda17b7 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:34.934472 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f804720e-fe8d-4eab-8a26-a9ab1bda17b7 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:34.966798 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929eaae8-29c7-48f0-9c70-df11a9c67d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:34.967396 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929eaae8-29c7-48f0-9c70-df11a9c67d04 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:34.967682 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929eaae8-29c7-48f0-9c70-df11a9c67d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:34.967959 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929eaae8-29c7-48f0-9c70-df11a9c67d04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:34.968738 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929eaae8-29c7-48f0-9c70-df11a9c67d04 None admin] RBAC: Adding query filter params (role.id=aded7bd43e974686b3c23bd9d4255b24, scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec, user.id=fe003cd7bfbe4b92b1d64d300c3e7d8b) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:34.969112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929eaae8-29c7-48f0-9c70-df11a9c67d04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:34.970574 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929eaae8-29c7-48f0-9c70-df11a9c67d04 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:34.976238 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 650/1287] 10.222.0.39 () {56 vars in 1374 bytes} [Thu Jul 28 22:18:34 2022] GET /identity/v3/role_assignments?user.id=fe003cd7bfbe4b92b1d64d300c3e7d8b&scope.project.id=b9fb2a1c6ad9453d9daf18d42a6f24ec&role.id=aded7bd43e974686b3c23bd9d4255b24 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:36.964727 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a87f12-75ff-4dd2-afd7-62297bd25a30 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:36.965257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a87f12-75ff-4dd2-afd7-62297bd25a30 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:36.965257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a87f12-75ff-4dd2-afd7-62297bd25a30 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:36.966659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 638/1288] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:18:36.976284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d16183-f05e-4bff-b78f-40b6eabbb3c0 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:36.976703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d16183-f05e-4bff-b78f-40b6eabbb3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:36.976886 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d16183-f05e-4bff-b78f-40b6eabbb3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:36.995674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d0d16183-f05e-4bff-b78f-40b6eabbb3c0 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:18:36.996840 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d0d16183-f05e-4bff-b78f-40b6eabbb3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:18:37.025074 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:37.063722 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 651/1289] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:18:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:18:37.085550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a87f12-75ff-4dd2-afd7-62297bd25a30 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:37.186902 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a87f12-75ff-4dd2-afd7-62297bd25a30 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:37.198064 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:37.221871 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfeb10-5f57-4e4a-a1b8-eda3cb9c3e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:37.222440 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfeb10-5f57-4e4a-a1b8-eda3cb9c3e9e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:37.222721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfeb10-5f57-4e4a-a1b8-eda3cb9c3e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:37.222996 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dfeb10-5f57-4e4a-a1b8-eda3cb9c3e9e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:37.223623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfeb10-5f57-4e4a-a1b8-eda3cb9c3e9e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:37.224981 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dfeb10-5f57-4e4a-a1b8-eda3cb9c3e9e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:37.233661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-13dfeb10-5f57-4e4a-a1b8-eda3cb9c3e9e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 22:18:37.233661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:37.233661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:18:37.233661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:18:37.233661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:18:37.233661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:18:37.233661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:18:37.233661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:18:37.233661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 22:18:37.233661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:37.233661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:18:37.233661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:37.233661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:37.233661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:37.233661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:37.233661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:37.233661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:37.236032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:37.236032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:37.236032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:37.236032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:37.236032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:37.236032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:37.236032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:18:37.236032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:18:37.236032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:18:37.236032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:18:37.236032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:37.236032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:37.236032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:37.236032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:37.236032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:37.236032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:37.237292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:37.237292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:37.237292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:37.237292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:37.237292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:37.237292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:37.237292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:37.237292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:37.237292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:37.237292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:37.237292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:18:37.237292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:18:37.237292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 22:18:37.237292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:37.237292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 639/1290] 10.222.0.39 () {56 vars in 1117 bytes} [Thu Jul 28 22:18:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:37.241641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d16183-f05e-4bff-b78f-40b6eabbb3c0 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:37.341782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d16183-f05e-4bff-b78f-40b6eabbb3c0 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:37.376214 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050046a0-106a-47ae-8f75-be1c5151222d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:37.376820 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050046a0-106a-47ae-8f75-be1c5151222d None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:37.377102 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050046a0-106a-47ae-8f75-be1c5151222d None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:37.377378 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050046a0-106a-47ae-8f75-be1c5151222d None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:37.378082 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050046a0-106a-47ae-8f75-be1c5151222d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:37.378389 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050046a0-106a-47ae-8f75-be1c5151222d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:37.379755 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050046a0-106a-47ae-8f75-be1c5151222d None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:37.384509 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 652/1291] 10.222.0.39 () {56 vars in 1116 bytes} [Thu Jul 28 22:18:37 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:37.390566 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfeb10-5f57-4e4a-a1b8-eda3cb9c3e9e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:37.493623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dfeb10-5f57-4e4a-a1b8-eda3cb9c3e9e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:37.527355 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd27e9c8-f140-46f9-b44a-7bf7a708f7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:37.527967 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27e9c8-f140-46f9-b44a-7bf7a708f7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:37.528249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27e9c8-f140-46f9-b44a-7bf7a708f7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:37.528522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd27e9c8-f140-46f9-b44a-7bf7a708f7f5 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:37.529215 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd27e9c8-f140-46f9-b44a-7bf7a708f7f5 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:37.529504 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd27e9c8-f140-46f9-b44a-7bf7a708f7f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:37.530889 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd27e9c8-f140-46f9-b44a-7bf7a708f7f5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:37.535756 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 640/1292] 10.222.0.39 () {56 vars in 1116 bytes} [Thu Jul 28 22:18:37 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:39.514403 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e592a-21d1-46bb-b0de-389fd32d7013 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:39.514943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e592a-21d1-46bb-b0de-389fd32d7013 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:39.514943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581e592a-21d1-46bb-b0de-389fd32d7013 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:39.516263 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 653/1293] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:18:39.521248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935f200-f77f-4f2e-81c0-6c2349403154 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:39.521707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935f200-f77f-4f2e-81c0-6c2349403154 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:39.521814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935f200-f77f-4f2e-81c0-6c2349403154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:39.539463 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c935f200-f77f-4f2e-81c0-6c2349403154 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:18:39.540642 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c935f200-f77f-4f2e-81c0-6c2349403154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:18:39.571276 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:39.608502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 641/1294] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:18:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:18:39.632121 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e592a-21d1-46bb-b0de-389fd32d7013 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:39.662603 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:39.731140 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581e592a-21d1-46bb-b0de-389fd32d7013 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:39.763767 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65838a-b4b2-4125-bca3-89190c71a58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:39.764361 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65838a-b4b2-4125-bca3-89190c71a58f None admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:39.764688 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65838a-b4b2-4125-bca3-89190c71a58f None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:39.764909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65838a-b4b2-4125-bca3-89190c71a58f None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:39.765669 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e65838a-b4b2-4125-bca3-89190c71a58f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:39.767615 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e65838a-b4b2-4125-bca3-89190c71a58f None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:39.781947 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 654/1295] 10.222.0.39 () {60 vars in 1141 bytes} [Thu Jul 28 22:18:39 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:18:41.798328 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92478f-7ee3-4a42-8eec-e10e36806b3a None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:41.798917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92478f-7ee3-4a42-8eec-e10e36806b3a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:41.799031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92478f-7ee3-4a42-8eec-e10e36806b3a None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:41.800444 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 642/1296] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:18:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:18:41.805655 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23085f-7d93-4c04-8267-82e34e7129b2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:41.806077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23085f-7d93-4c04-8267-82e34e7129b2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:41.806224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e23085f-7d93-4c04-8267-82e34e7129b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:41.825694 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3e23085f-7d93-4c04-8267-82e34e7129b2 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:18:41.826914 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3e23085f-7d93-4c04-8267-82e34e7129b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:18:41.910680 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 655/1297] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:18:41 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:18:41.933117 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92478f-7ee3-4a42-8eec-e10e36806b3a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:42.032796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92478f-7ee3-4a42-8eec-e10e36806b3a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:42.065650 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b5098-3e8f-4caa-99b7-0305176cb683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:42.066210 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b5098-3e8f-4caa-99b7-0305176cb683 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:42.066488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b5098-3e8f-4caa-99b7-0305176cb683 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:42.066759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b5098-3e8f-4caa-99b7-0305176cb683 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:42.067382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b5098-3e8f-4caa-99b7-0305176cb683 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:42.068729 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b5098-3e8f-4caa-99b7-0305176cb683 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:42.077252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-132b5098-3e8f-4caa-99b7-0305176cb683 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 22:18:42.077252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:42.077252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:18:42.077252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:18:42.077252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:18:42.077252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:18:42.077252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:18:42.077252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:18:42.077252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 22:18:42.077252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:42.077252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:18:42.077252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:42.077252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:42.077252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:42.077252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:42.077252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:42.077252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:42.079296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:42.079296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:42.079296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:42.079296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:42.079296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:42.079296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:42.079296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:18:42.079296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:18:42.079296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:18:42.079296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:18:42.079296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:42.079296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:42.079296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:42.079296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:42.079296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:42.079296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:42.080442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:42.080442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:42.080442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:42.080442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:42.080442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:42.080442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:42.080442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:42.080442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:42.080442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:42.080442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:42.080442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:18:42.080442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:18:42.080442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 22:18:42.080442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:42.080442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 643/1298] 10.222.0.39 () {56 vars in 1117 bytes} [Thu Jul 28 22:18:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:42.085120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23085f-7d93-4c04-8267-82e34e7129b2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:42.102878 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:42.184087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e23085f-7d93-4c04-8267-82e34e7129b2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:42.217570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823bd96-f73b-4138-9c34-431a3eff1026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:42.218186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823bd96-f73b-4138-9c34-431a3eff1026 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:42.218467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823bd96-f73b-4138-9c34-431a3eff1026 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:42.218818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823bd96-f73b-4138-9c34-431a3eff1026 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:42.219507 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823bd96-f73b-4138-9c34-431a3eff1026 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:42.219809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823bd96-f73b-4138-9c34-431a3eff1026 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:42.221199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823bd96-f73b-4138-9c34-431a3eff1026 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:42.226247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 656/1299] 10.222.0.39 () {56 vars in 1116 bytes} [Thu Jul 28 22:18:42 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:42.232326 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b5098-3e8f-4caa-99b7-0305176cb683 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:42.282480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:42.366633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b5098-3e8f-4caa-99b7-0305176cb683 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:42.398748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950902e1-8307-4bf2-aab5-c34bc74ffb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:42.399352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950902e1-8307-4bf2-aab5-c34bc74ffb89 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:42.399628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950902e1-8307-4bf2-aab5-c34bc74ffb89 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:42.399895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950902e1-8307-4bf2-aab5-c34bc74ffb89 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:42.400635 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950902e1-8307-4bf2-aab5-c34bc74ffb89 None admin] RBAC: Adding query filter params (interface=public, service_id=4b16e9b72bc748279b3b682c858e7cf9, region_id=RegionOne) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:42.400938 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950902e1-8307-4bf2-aab5-c34bc74ffb89 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:42.402312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950902e1-8307-4bf2-aab5-c34bc74ffb89 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:42.407356 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 644/1300] 10.222.0.39 () {56 vars in 1251 bytes} [Thu Jul 28 22:18:42 2022] GET /identity/v3/endpoints?service_id=4b16e9b72bc748279b3b682c858e7cf9&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:18:42.412985 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823bd96-f73b-4138-9c34-431a3eff1026 None admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:42.491937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:42.510654 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823bd96-f73b-4138-9c34-431a3eff1026 None admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:42.542959 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5f9c7-cdbc-4c12-8bc0-42731dfba3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:42.543585 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5f9c7-cdbc-4c12-8bc0-42731dfba3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:42.543857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5f9c7-cdbc-4c12-8bc0-42731dfba3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:42.544133 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5f9c7-cdbc-4c12-8bc0-42731dfba3c0 None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:42.544784 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e5f9c7-cdbc-4c12-8bc0-42731dfba3c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:42.546194 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e5f9c7-cdbc-4c12-8bc0-42731dfba3c0 None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:42.553651 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 657/1301] 10.222.0.39 () {56 vars in 1087 bytes} [Thu Jul 28 22:18:42 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:18:44.552392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9997e5e-3daa-44c8-8d73-3527d944fb79 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:44.552908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9997e5e-3daa-44c8-8d73-3527d944fb79 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:44.552908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9997e5e-3daa-44c8-8d73-3527d944fb79 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:44.554224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 645/1302] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:18:44.559173 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02f8e5-70b4-4a48-a5b4-ff61e08bfa61 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:44.559610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02f8e5-70b4-4a48-a5b4-ff61e08bfa61 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:44.559740 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02f8e5-70b4-4a48-a5b4-ff61e08bfa61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:44.579060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9d02f8e5-70b4-4a48-a5b4-ff61e08bfa61 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:18:44.580245 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9d02f8e5-70b4-4a48-a5b4-ff61e08bfa61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:18:44.645176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 658/1303] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:18:44 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 28 22:18:44.667424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9997e5e-3daa-44c8-8d73-3527d944fb79 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:44.760595 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:44.767483 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9997e5e-3daa-44c8-8d73-3527d944fb79 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:44.801341 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6de9d7-9a22-48a9-93a4-8b140d8f3fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:44.801997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6de9d7-9a22-48a9-93a4-8b140d8f3fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:44.802276 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6de9d7-9a22-48a9-93a4-8b140d8f3fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:44.802617 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6de9d7-9a22-48a9-93a4-8b140d8f3fa4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:44.803266 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6de9d7-9a22-48a9-93a4-8b140d8f3fa4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:44.804773 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6de9d7-9a22-48a9-93a4-8b140d8f3fa4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:44.816046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1b6de9d7-9a22-48a9-93a4-8b140d8f3fa4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 22:18:44.816046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:44.816046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 28 22:18:44.816046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 28 22:18:44.816046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 28 22:18:44.816046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 28 22:18:44.816046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 28 22:18:44.816046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:18:44.816046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 22:18:44.816046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:44.816046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 28 22:18:44.816046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:44.816046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 28 22:18:44.816046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 28 22:18:44.816046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 28 22:18:44.816046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 28 22:18:44.816046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 28 22:18:44.817082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 28 22:18:44.817082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 28 22:18:44.817082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 28 22:18:44.817082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 28 22:18:44.817082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 28 22:18:44.817082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 28 22:18:44.817082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 28 22:18:44.817082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 28 22:18:44.817082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 28 22:18:44.817082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 28 22:18:44.817082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 28 22:18:44.817082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 28 22:18:44.817082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 28 22:18:44.817082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 28 22:18:44.817082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 28 22:18:44.817082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jul 28 22:18:44.818240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 28 22:18:44.818240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jul 28 22:18:44.818240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 28 22:18:44.818240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jul 28 22:18:44.818240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 28 22:18:44.818240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 28 22:18:44.818240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 28 22:18:44.818240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jul 28 22:18:44.818240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 28 22:18:44.818240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jul 28 22:18:44.818240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 28 22:18:44.818240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 28 22:18:44.818240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 28 22:18:44.818240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jul 28 22:18:44.818240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 646/1304] 10.222.0.39 () {56 vars in 1117 bytes} [Thu Jul 28 22:18:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 28 22:18:44.823918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02f8e5-70b4-4a48-a5b4-ff61e08bfa61 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:44.903933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:44.926883 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02f8e5-70b4-4a48-a5b4-ff61e08bfa61 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:44.960059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4eb8ba-6e47-404f-bd25-8b8c4f14e9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:44.960669 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4eb8ba-6e47-404f-bd25-8b8c4f14e9ed None admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:44.960950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4eb8ba-6e47-404f-bd25-8b8c4f14e9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:44.961223 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4eb8ba-6e47-404f-bd25-8b8c4f14e9ed None admin] PATH_INFO: `/v3/services` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:44.961923 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4eb8ba-6e47-404f-bd25-8b8c4f14e9ed None admin] RBAC: Adding query filter params (name=placement) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:18:44.962225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4eb8ba-6e47-404f-bd25-8b8c4f14e9ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:44.963615 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4eb8ba-6e47-404f-bd25-8b8c4f14e9ed None admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:44.969065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 659/1305] 10.222.0.39 () {56 vars in 1116 bytes} [Thu Jul 28 22:18:44 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:18:44.974895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6de9d7-9a22-48a9-93a4-8b140d8f3fa4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:18:45.072680 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6de9d7-9a22-48a9-93a4-8b140d8f3fa4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:18:45.105519 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f0ef1-f9c5-436d-a30a-419955799261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:18:45.106143 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f0ef1-f9c5-436d-a30a-419955799261 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:18:45.106421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f0ef1-f9c5-436d-a30a-419955799261 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:18:45.106692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f0ef1-f9c5-436d-a30a-419955799261 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:18:45.107389 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f0ef1-f9c5-436d-a30a-419955799261 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:18:45.108743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f0ef1-f9c5-436d-a30a-419955799261 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:18:45.127957 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 647/1306] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:18:44 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:18:54.966040 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=30 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:18:55.123808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:19:48.377996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55a3b2-d9eb-4225-a03a-9a7bc708ce31 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:19:48.378538 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55a3b2-d9eb-4225-a03a-9a7bc708ce31 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:19:48.378538 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55a3b2-d9eb-4225-a03a-9a7bc708ce31 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:19:48.385300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49f865-d15b-45d8-b1b1-fcdef6959578 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:19:48.389596 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49f865-d15b-45d8-b1b1-fcdef6959578 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:19:48.389596 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49f865-d15b-45d8-b1b1-fcdef6959578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:19:48.390937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 660/1307] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:19:48 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:19:48.467398 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-fb49f865-d15b-45d8-b1b1-fcdef6959578 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:19:48.468581 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fb49f865-d15b-45d8-b1b1-fcdef6959578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:19:48.583429 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 648/1308] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:19:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:19:48.633339 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c654bc-b7b0-4b36-a7f1-d863bb5ec3cd None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:19:48.633659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c654bc-b7b0-4b36-a7f1-d863bb5ec3cd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:19:48.633932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c654bc-b7b0-4b36-a7f1-d863bb5ec3cd None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:19:48.635292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 661/1309] 10.222.0.39 () {54 vars in 965 bytes} [Thu Jul 28 22:19:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:19:48.648050 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a541bd-eb8a-427c-9440-97b4364ba6d3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:19:48.649019 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a541bd-eb8a-427c-9440-97b4364ba6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:19:48.649937 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a541bd-eb8a-427c-9440-97b4364ba6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:19:48.780591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-99a541bd-eb8a-427c-9440-97b4364ba6d3 None None] MFA Rules not processed for user `09b96431ac98447283960019cadde8a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:19:48.782918 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-99a541bd-eb8a-427c-9440-97b4364ba6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:19:48.808022 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:19:48.867739 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 649/1310] 10.222.0.39 () {58 vars in 1062 bytes} [Thu Jul 28 22:19:48 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:19:48.877275 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c654bc-b7b0-4b36-a7f1-d863bb5ec3cd None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:19:48.879117 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-70c654bc-b7b0-4b36-a7f1-d863bb5ec3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:19:48.998323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c654bc-b7b0-4b36-a7f1-d863bb5ec3cd None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:19:49.080424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535e552-37ad-41ce-8eff-51d960bae610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:19:49.080986 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535e552-37ad-41ce-8eff-51d960bae610 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:19:49.081267 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535e552-37ad-41ce-8eff-51d960bae610 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:19:49.081552 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535e552-37ad-41ce-8eff-51d960bae610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:19:49.083710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f535e552-37ad-41ce-8eff-51d960bae610 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:19:49.158977 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:19:49.218371 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535e552-37ad-41ce-8eff-51d960bae610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:19:49.220512 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535e552-37ad-41ce-8eff-51d960bae610 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:19:49.241171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 662/1311] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:19:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:19:51.568634 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5cec1d-68ed-443b-9ecb-63c854d3e94d None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:19:51.569239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5cec1d-68ed-443b-9ecb-63c854d3e94d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:19:51.569239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5cec1d-68ed-443b-9ecb-63c854d3e94d None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:19:51.571216 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 650/1312] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:19:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:19:51.576327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48feffb-1ec8-4da6-b514-65e5364a4d9d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:19:51.576863 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48feffb-1ec8-4da6-b514-65e5364a4d9d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:19:51.576863 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48feffb-1ec8-4da6-b514-65e5364a4d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:19:51.671398 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d48feffb-1ec8-4da6-b514-65e5364a4d9d None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:19:51.673196 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d48feffb-1ec8-4da6-b514-65e5364a4d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:19:51.784926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 663/1313] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:19:51.823733 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5cec1d-68ed-443b-9ecb-63c854d3e94d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:19:51.825511 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0d5cec1d-68ed-443b-9ecb-63c854d3e94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:19:51.940587 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5cec1d-68ed-443b-9ecb-63c854d3e94d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:19:52.014096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:19:52.017549 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5be0a0-4f47-4c8e-8272-59467a2e4794 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:19:52.018146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5be0a0-4f47-4c8e-8272-59467a2e4794 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:19:52.018391 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5be0a0-4f47-4c8e-8272-59467a2e4794 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:19:52.018789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5be0a0-4f47-4c8e-8272-59467a2e4794 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:19:52.020991 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7f5be0a0-4f47-4c8e-8272-59467a2e4794 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:19:52.133230 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5be0a0-4f47-4c8e-8272-59467a2e4794 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:19:52.134702 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5be0a0-4f47-4c8e-8272-59467a2e4794 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:19:52.154251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 651/1314] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:19:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:19:54.456352 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731292c2-77e7-4916-a34b-feaff17fc423 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:19:54.456896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731292c2-77e7-4916-a34b-feaff17fc423 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:19:54.456896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731292c2-77e7-4916-a34b-feaff17fc423 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:19:54.458298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 664/1315] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:19:54.463473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb01f68-0f91-4d7f-ae48-3a9e483736bd None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:19:54.463859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb01f68-0f91-4d7f-ae48-3a9e483736bd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:19:54.464024 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb01f68-0f91-4d7f-ae48-3a9e483736bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:19:54.530076 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4fb01f68-0f91-4d7f-ae48-3a9e483736bd None None] MFA Rules not processed for user `f7fc4b88abc344d2b72a4653dc811819`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:19:54.531909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4fb01f68-0f91-4d7f-ae48-3a9e483736bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:19:54.584263 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:19:54.629805 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 652/1316] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:19:54 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:19:54.669928 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731292c2-77e7-4916-a34b-feaff17fc423 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:19:54.671959 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-731292c2-77e7-4916-a34b-feaff17fc423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:19:54.681787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:19:54.785955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731292c2-77e7-4916-a34b-feaff17fc423 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:19:54.856826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ef1c82-1847-4c61-be74-32482f8ca6ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:19:54.857439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef1c82-1847-4c61-be74-32482f8ca6ad service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:19:54.857742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef1c82-1847-4c61-be74-32482f8ca6ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:19:54.858012 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef1c82-1847-4c61-be74-32482f8ca6ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:19:54.860284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b8ef1c82-1847-4c61-be74-32482f8ca6ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:19:54.919719 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:19:54.966313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef1c82-1847-4c61-be74-32482f8ca6ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:19:54.967746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ef1c82-1847-4c61-be74-32482f8ca6ad service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:19:54.987745 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 665/1317] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:19:54 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:19:57.557844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bd802-048f-4639-bf4d-cbd5aefd038f None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:19:57.558357 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bd802-048f-4639-bf4d-cbd5aefd038f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:19:57.558357 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bd802-048f-4639-bf4d-cbd5aefd038f None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:19:57.559669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 653/1318] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:19:57.565180 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8e86b-a5ce-4621-a657-0be4d5b2b659 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:19:57.565596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8e86b-a5ce-4621-a657-0be4d5b2b659 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:19:57.565771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8e86b-a5ce-4621-a657-0be4d5b2b659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:19:57.630212 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b0c8e86b-a5ce-4621-a657-0be4d5b2b659 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:19:57.631420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b0c8e86b-a5ce-4621-a657-0be4d5b2b659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:19:57.726649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 666/1319] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:19:57 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:19:57.765959 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bd802-048f-4639-bf4d-cbd5aefd038f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:19:57.767538 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4b7bd802-048f-4639-bf4d-cbd5aefd038f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:19:57.872985 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bd802-048f-4639-bf4d-cbd5aefd038f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:19:57.945417 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c08da3-91b2-4fcf-b5f9-4f92a123aea7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:19:57.946063 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c08da3-91b2-4fcf-b5f9-4f92a123aea7 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:19:57.946346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c08da3-91b2-4fcf-b5f9-4f92a123aea7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:19:57.946620 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c08da3-91b2-4fcf-b5f9-4f92a123aea7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:19:57.948232 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-93c08da3-91b2-4fcf-b5f9-4f92a123aea7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:19:57.956185 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:19:58.056787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c08da3-91b2-4fcf-b5f9-4f92a123aea7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:19:58.058257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c08da3-91b2-4fcf-b5f9-4f92a123aea7 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:19:58.076427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 654/1320] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:19:57 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:20:00.748487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bb843-04e2-4548-b685-55d27de1c68a None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:00.749031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bb843-04e2-4548-b685-55d27de1c68a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:00.749031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6bb843-04e2-4548-b685-55d27de1c68a None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:00.750580 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 667/1321] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:00.756144 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5059bf-3b37-4efd-a2bf-b032c0f6a870 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:00.756572 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5059bf-3b37-4efd-a2bf-b032c0f6a870 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:00.756704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5059bf-3b37-4efd-a2bf-b032c0f6a870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:00.825213 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-cd5059bf-3b37-4efd-a2bf-b032c0f6a870 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:00.826487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cd5059bf-3b37-4efd-a2bf-b032c0f6a870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:00.884745 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:00.927537 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 655/1322] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:20:00.980458 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45f9d7-0edb-4d14-ae51-ee79edd26242 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:00.980749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45f9d7-0edb-4d14-ae51-ee79edd26242 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:00.981026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45f9d7-0edb-4d14-ae51-ee79edd26242 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:00.982513 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 668/1323] 10.222.0.39 () {54 vars in 965 bytes} [Thu Jul 28 22:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:20:00.989728 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a66593-1a7e-4047-8c7a-e999a6950c91 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:00.990024 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a66593-1a7e-4047-8c7a-e999a6950c91 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:00.990303 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a66593-1a7e-4047-8c7a-e999a6950c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:01.073636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b6a66593-1a7e-4047-8c7a-e999a6950c91 None None] MFA Rules not processed for user `09b96431ac98447283960019cadde8a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:01.074834 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b6a66593-1a7e-4047-8c7a-e999a6950c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:01.156471 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 656/1324] 10.222.0.39 () {58 vars in 1062 bytes} [Thu Jul 28 22:20:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:20:01.165571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45f9d7-0edb-4d14-ae51-ee79edd26242 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:20:01.167105 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0a45f9d7-0edb-4d14-ae51-ee79edd26242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:01.187865 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:01.275844 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45f9d7-0edb-4d14-ae51-ee79edd26242 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:20:01.348174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21994f-49bd-4057-a8b3-a5617a479ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:20:01.348795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21994f-49bd-4057-a8b3-a5617a479ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:01.349076 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21994f-49bd-4057-a8b3-a5617a479ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:01.349350 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21994f-49bd-4057-a8b3-a5617a479ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:01.351012 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-df21994f-49bd-4057-a8b3-a5617a479ee4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:01.424610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:01.463153 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21994f-49bd-4057-a8b3-a5617a479ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:20:01.464627 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21994f-49bd-4057-a8b3-a5617a479ee4 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:20:01.483622 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 669/1325] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:20:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:20:03.801689 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3c936-93d3-4518-86e7-64b78e6cf83d None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:03.802246 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3c936-93d3-4518-86e7-64b78e6cf83d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:03.802246 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc3c936-93d3-4518-86e7-64b78e6cf83d None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:03.804054 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 657/1326] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:20:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:03.811848 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52e9de-53fb-47a3-ade3-8aff966bd4fd None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:03.811848 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52e9de-53fb-47a3-ade3-8aff966bd4fd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:03.811848 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce52e9de-53fb-47a3-ade3-8aff966bd4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:03.900463 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ce52e9de-53fb-47a3-ade3-8aff966bd4fd None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:03.902883 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ce52e9de-53fb-47a3-ade3-8aff966bd4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:04.044005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 670/1327] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:20:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:20:04.083322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3c936-93d3-4518-86e7-64b78e6cf83d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:20:04.085249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2dc3c936-93d3-4518-86e7-64b78e6cf83d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:04.156782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:04.213238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc3c936-93d3-4518-86e7-64b78e6cf83d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:20:04.293614 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb70ff2-17d3-4ff1-9ce5-e1c3d12d97b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:20:04.294164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb70ff2-17d3-4ff1-9ce5-e1c3d12d97b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:04.294447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb70ff2-17d3-4ff1-9ce5-e1c3d12d97b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:04.295360 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb70ff2-17d3-4ff1-9ce5-e1c3d12d97b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:04.298060 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ccb70ff2-17d3-4ff1-9ce5-e1c3d12d97b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:04.418055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:04.420678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb70ff2-17d3-4ff1-9ce5-e1c3d12d97b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:20:04.424580 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb70ff2-17d3-4ff1-9ce5-e1c3d12d97b7 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:20:04.453864 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 658/1328] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:20:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:20:06.525360 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0b466-4529-4484-a0ec-d39b626f1add None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:06.525913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0b466-4529-4484-a0ec-d39b626f1add None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:06.525913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f0b466-4529-4484-a0ec-d39b626f1add None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:06.527223 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 671/1329] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:20:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:06.533895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c329f21-77db-4874-b1c1-93fa094f4ef0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:06.534423 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c329f21-77db-4874-b1c1-93fa094f4ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:06.534574 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c329f21-77db-4874-b1c1-93fa094f4ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:06.601920 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4c329f21-77db-4874-b1c1-93fa094f4ef0 None None] MFA Rules not processed for user `f7fc4b88abc344d2b72a4653dc811819`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:06.603151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4c329f21-77db-4874-b1c1-93fa094f4ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:06.709494 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 659/1330] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:20:06 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:20:06.743057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0b466-4529-4484-a0ec-d39b626f1add None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:20:06.744602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-69f0b466-4529-4484-a0ec-d39b626f1add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:06.780122 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:06.857146 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f0b466-4529-4484-a0ec-d39b626f1add None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:20:06.931667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489c01f-b0fa-483e-8f32-f013ed274e8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:20:06.932295 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489c01f-b0fa-483e-8f32-f013ed274e8d service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:06.932582 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489c01f-b0fa-483e-8f32-f013ed274e8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:06.932861 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489c01f-b0fa-483e-8f32-f013ed274e8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:06.934531 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a489c01f-b0fa-483e-8f32-f013ed274e8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:07.013627 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:07.047798 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489c01f-b0fa-483e-8f32-f013ed274e8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:20:07.049168 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489c01f-b0fa-483e-8f32-f013ed274e8d service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:20:07.067571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 672/1331] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:20:06 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:20:09.497570 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97802a8-6293-485e-8eff-c1d85ea65eb7 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:09.498077 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97802a8-6293-485e-8eff-c1d85ea65eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:09.498077 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97802a8-6293-485e-8eff-c1d85ea65eb7 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:09.500031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 660/1332] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:20:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:09.505019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b009d1-5147-45b5-8829-9ef44f400b47 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:09.505419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b009d1-5147-45b5-8829-9ef44f400b47 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:09.505603 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b009d1-5147-45b5-8829-9ef44f400b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:09.576801 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-56b009d1-5147-45b5-8829-9ef44f400b47 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:09.578058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-56b009d1-5147-45b5-8829-9ef44f400b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:09.693665 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 673/1333] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:20:09.729825 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97802a8-6293-485e-8eff-c1d85ea65eb7 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:20:09.731346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f97802a8-6293-485e-8eff-c1d85ea65eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:09.802241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:09.836804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97802a8-6293-485e-8eff-c1d85ea65eb7 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:20:09.911507 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b661bb-821e-4b38-85e3-7b338676f00b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:20:09.912390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b661bb-821e-4b38-85e3-7b338676f00b service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:09.912673 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b661bb-821e-4b38-85e3-7b338676f00b service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:09.912943 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b661bb-821e-4b38-85e3-7b338676f00b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:09.914574 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-62b661bb-821e-4b38-85e3-7b338676f00b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:10.022328 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b661bb-821e-4b38-85e3-7b338676f00b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:20:10.023768 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b661bb-821e-4b38-85e3-7b338676f00b service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:20:10.034314 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:10.043105 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 661/1334] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:20:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:20:12.141260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae54d0d-8c15-4acb-9a5d-589f511b6ba1 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:12.141847 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae54d0d-8c15-4acb-9a5d-589f511b6ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:12.141847 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae54d0d-8c15-4acb-9a5d-589f511b6ba1 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:12.143291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 674/1335] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:20:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:12.148757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530ca90-ef2f-4456-9b76-0c47a3a7438f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:12.149200 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530ca90-ef2f-4456-9b76-0c47a3a7438f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:12.149309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530ca90-ef2f-4456-9b76-0c47a3a7438f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:12.215714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5530ca90-ef2f-4456-9b76-0c47a3a7438f None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:12.216922 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5530ca90-ef2f-4456-9b76-0c47a3a7438f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:12.316413 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 662/1336] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:20:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:20:12.355923 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae54d0d-8c15-4acb-9a5d-589f511b6ba1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:20:12.357471 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0ae54d0d-8c15-4acb-9a5d-589f511b6ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:12.399673 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:12.465668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae54d0d-8c15-4acb-9a5d-589f511b6ba1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:20:12.539071 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df0132-3a2e-46ff-806b-4cd3acc4ce5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:20:12.539688 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df0132-3a2e-46ff-806b-4cd3acc4ce5a service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:12.539972 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df0132-3a2e-46ff-806b-4cd3acc4ce5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:12.540297 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df0132-3a2e-46ff-806b-4cd3acc4ce5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:12.541915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-89df0132-3a2e-46ff-806b-4cd3acc4ce5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:12.625370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:12.650878 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df0132-3a2e-46ff-806b-4cd3acc4ce5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:20:12.652311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df0132-3a2e-46ff-806b-4cd3acc4ce5a service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:20:12.669910 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 675/1337] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:20:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:20:15.021663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc037d-26b9-4ed6-9895-28adb89db26c None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:15.022213 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc037d-26b9-4ed6-9895-28adb89db26c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:15.022213 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cc037d-26b9-4ed6-9895-28adb89db26c None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:15.023580 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 663/1338] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:20:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:15.028527 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2557b4-2ede-4f55-98ba-59b13def2f56 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:15.028922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2557b4-2ede-4f55-98ba-59b13def2f56 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:15.029080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2557b4-2ede-4f55-98ba-59b13def2f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:15.093764 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-2e2557b4-2ede-4f55-98ba-59b13def2f56 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:15.094968 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2e2557b4-2ede-4f55-98ba-59b13def2f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:15.198720 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 676/1339] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:20:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:20:15.234898 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc037d-26b9-4ed6-9895-28adb89db26c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:20:15.236427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-11cc037d-26b9-4ed6-9895-28adb89db26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:15.310740 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:15.340242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cc037d-26b9-4ed6-9895-28adb89db26c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:20:15.415217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec918b-a946-4a0b-9cd7-6f8845281c9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:20:15.415762 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec918b-a946-4a0b-9cd7-6f8845281c9e service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:15.416046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec918b-a946-4a0b-9cd7-6f8845281c9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:15.416324 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec918b-a946-4a0b-9cd7-6f8845281c9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:15.417970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-96ec918b-a946-4a0b-9cd7-6f8845281c9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:15.540507 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec918b-a946-4a0b-9cd7-6f8845281c9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:20:15.541973 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ec918b-a946-4a0b-9cd7-6f8845281c9e service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:20:15.559857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 664/1340] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:20:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:20:19.534805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa7c36-51b6-47b5-aeb1-011461674adc None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:19.535312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa7c36-51b6-47b5-aeb1-011461674adc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:19.535312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aa7c36-51b6-47b5-aeb1-011461674adc None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:19.536760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 677/1341] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:19.542375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a223590-e569-4f94-b657-358a322c3b4c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:19.542836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a223590-e569-4f94-b657-358a322c3b4c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:19.542934 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a223590-e569-4f94-b657-358a322c3b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:19.549870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:19.614709 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9a223590-e569-4f94-b657-358a322c3b4c None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:19.615923 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9a223590-e569-4f94-b657-358a322c3b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:19.715903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 665/1342] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:20:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:20:19.755131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa7c36-51b6-47b5-aeb1-011461674adc None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:20:19.756659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c0aa7c36-51b6-47b5-aeb1-011461674adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:19.806661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:19.863304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aa7c36-51b6-47b5-aeb1-011461674adc None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:20:19.937663 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d3912-a3f9-4018-b107-95342b15e2d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:20:19.938287 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d3912-a3f9-4018-b107-95342b15e2d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:19.938565 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d3912-a3f9-4018-b107-95342b15e2d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:19.938853 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d3912-a3f9-4018-b107-95342b15e2d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:19.940448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-350d3912-a3f9-4018-b107-95342b15e2d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:20.033962 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:20.052352 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d3912-a3f9-4018-b107-95342b15e2d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:20:20.053837 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d3912-a3f9-4018-b107-95342b15e2d0 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:20:20.074543 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 678/1343] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:20:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:20:22.716494 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1211653-3ebb-44cb-9c3e-107bfaceb305 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:22.717047 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1211653-3ebb-44cb-9c3e-107bfaceb305 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:22.717047 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1211653-3ebb-44cb-9c3e-107bfaceb305 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:22.718675 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 666/1344] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:22.724152 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f87873-7b42-43d5-9fe9-b08cf93f43d3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:22.724660 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f87873-7b42-43d5-9fe9-b08cf93f43d3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:22.724773 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f87873-7b42-43d5-9fe9-b08cf93f43d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:22.796956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a1f87873-7b42-43d5-9fe9-b08cf93f43d3 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:22.798232 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a1f87873-7b42-43d5-9fe9-b08cf93f43d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:22.901296 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 679/1345] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:20:22 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:20:22.939185 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1211653-3ebb-44cb-9c3e-107bfaceb305 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:20:22.940852 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f1211653-3ebb-44cb-9c3e-107bfaceb305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:23.029473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:23.047355 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1211653-3ebb-44cb-9c3e-107bfaceb305 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:20:23.121325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad43ee7-e082-4480-b708-b7661e767eb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:20:23.122029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad43ee7-e082-4480-b708-b7661e767eb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:23.122333 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad43ee7-e082-4480-b708-b7661e767eb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:23.122631 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad43ee7-e082-4480-b708-b7661e767eb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:23.124408 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3ad43ee7-e082-4480-b708-b7661e767eb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:23.235543 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad43ee7-e082-4480-b708-b7661e767eb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:20:23.237018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad43ee7-e082-4480-b708-b7661e767eb3 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:20:23.255535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 667/1346] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:20:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:20:27.273143 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea617c-ac88-4116-997f-39fd82ddaa0f None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:27.273698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea617c-ac88-4116-997f-39fd82ddaa0f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:27.273698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ea617c-ac88-4116-997f-39fd82ddaa0f None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:27.274994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 680/1347] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:27.280735 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e027dc-d324-42ea-bb04-0b2ad58f8d52 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:27.281247 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e027dc-d324-42ea-bb04-0b2ad58f8d52 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:27.281368 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e027dc-d324-42ea-bb04-0b2ad58f8d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:27.297115 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:27.351062 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-56e027dc-d324-42ea-bb04-0b2ad58f8d52 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:27.352271 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-56e027dc-d324-42ea-bb04-0b2ad58f8d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:27.450126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 668/1348] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:20:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:20:27.488643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea617c-ac88-4116-997f-39fd82ddaa0f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:20:27.490258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-43ea617c-ac88-4116-997f-39fd82ddaa0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:27.557898 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:27.625946 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ea617c-ac88-4116-997f-39fd82ddaa0f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:20:27.709084 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5812b-abc5-4842-ae1f-89d50d503ef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:20:27.709736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5812b-abc5-4842-ae1f-89d50d503ef5 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:27.710014 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5812b-abc5-4842-ae1f-89d50d503ef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:27.710282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5812b-abc5-4842-ae1f-89d50d503ef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:27.711917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-46b5812b-abc5-4842-ae1f-89d50d503ef5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:27.835352 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:27.839430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5812b-abc5-4842-ae1f-89d50d503ef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:20:27.840881 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5812b-abc5-4842-ae1f-89d50d503ef5 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:20:27.863811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 681/1349] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:20:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:20:32.767746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b88b4-b587-4e80-936f-fadcb0debd49 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:32.768228 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b88b4-b587-4e80-936f-fadcb0debd49 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:32.768228 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b88b4-b587-4e80-936f-fadcb0debd49 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:32.769757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 669/1350] 10.222.0.39 () {54 vars in 921 bytes} [Thu Jul 28 22:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:32.776522 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89997898-1660-4104-ad9c-f0e7ad38f515 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:32.777241 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89997898-1660-4104-ad9c-f0e7ad38f515 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:32.777869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89997898-1660-4104-ad9c-f0e7ad38f515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:33.034549 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-89997898-1660-4104-ad9c-f0e7ad38f515 None None] MFA Rules not processed for user `fe003cd7bfbe4b92b1d64d300c3e7d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:33.035911 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-89997898-1660-4104-ad9c-f0e7ad38f515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:33.172897 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 682/1351] 10.222.0.39 () {58 vars in 1018 bytes} [Thu Jul 28 22:20:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:20:37.349707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d0f3f-6802-487a-ab46-02a3936db7e5 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:37.350220 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d0f3f-6802-487a-ab46-02a3936db7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:37.350303 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d0f3f-6802-487a-ab46-02a3936db7e5 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:37.351859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 670/1352] 10.222.0.39 () {54 vars in 921 bytes} [Thu Jul 28 22:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:37.356924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd298b-04dd-47d6-9c24-9d4c60b27131 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:37.357348 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd298b-04dd-47d6-9c24-9d4c60b27131 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:37.357598 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd298b-04dd-47d6-9c24-9d4c60b27131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:37.414009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:37.438895 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-28bd298b-04dd-47d6-9c24-9d4c60b27131 None None] MFA Rules not processed for user `fe003cd7bfbe4b92b1d64d300c3e7d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:37.443070 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-28bd298b-04dd-47d6-9c24-9d4c60b27131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:37.448170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=59 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:37.569445 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 683/1353] 10.222.0.39 () {58 vars in 1018 bytes} [Thu Jul 28 22:20:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:20:38.906108 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae9101-8590-4a5c-acac-efa634a6b3c0 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:38.906980 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae9101-8590-4a5c-acac-efa634a6b3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:38.907266 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae9101-8590-4a5c-acac-efa634a6b3c0 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:38.908634 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 671/1354] 10.222.0.39 () {54 vars in 921 bytes} [Thu Jul 28 22:20:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:38.913725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2668ad-c399-4286-88bd-28cda6073139 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:38.914160 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2668ad-c399-4286-88bd-28cda6073139 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:38.914481 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2668ad-c399-4286-88bd-28cda6073139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:38.994441 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-0e2668ad-c399-4286-88bd-28cda6073139 None None] MFA Rules not processed for user `fe003cd7bfbe4b92b1d64d300c3e7d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:38.995859 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0e2668ad-c399-4286-88bd-28cda6073139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:39.042891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:39.110429 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 684/1355] 10.222.0.39 () {58 vars in 1018 bytes} [Thu Jul 28 22:20:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:20:42.476383 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddedb75-1fc6-45de-b3de-58c8931523b1 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:42.476951 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddedb75-1fc6-45de-b3de-58c8931523b1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:42.476951 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddedb75-1fc6-45de-b3de-58c8931523b1 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:42.480912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 672/1356] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:20:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:42.486796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aed040-ec52-45f8-bb12-aa874cec247c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:42.487736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aed040-ec52-45f8-bb12-aa874cec247c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:42.488241 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aed040-ec52-45f8-bb12-aa874cec247c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:42.583586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-53aed040-ec52-45f8-bb12-aa874cec247c None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:42.585203 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-53aed040-ec52-45f8-bb12-aa874cec247c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:42.696081 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 685/1357] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:20:42.752506 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80730a62-a4f1-4f4a-9a78-9518cdd0f092 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:42.752778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80730a62-a4f1-4f4a-9a78-9518cdd0f092 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:42.753412 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80730a62-a4f1-4f4a-9a78-9518cdd0f092 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:42.755362 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 673/1358] 10.222.0.39 () {54 vars in 962 bytes} [Thu Jul 28 22:20:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:42.763424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c40948-3764-4100-bf4e-d2aa37dbe311 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:42.764199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c40948-3764-4100-bf4e-d2aa37dbe311 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:42.765476 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c40948-3764-4100-bf4e-d2aa37dbe311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:42.786942 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:42.852296 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-23c40948-3764-4100-bf4e-d2aa37dbe311 None None] MFA Rules not processed for user `77efe8046ea0468cb0c7c58c24fc4e75`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:42.853491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-23c40948-3764-4100-bf4e-d2aa37dbe311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:42.987474 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 686/1359] 10.222.0.39 () {58 vars in 1059 bytes} [Thu Jul 28 22:20:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:20:42.995615 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80730a62-a4f1-4f4a-9a78-9518cdd0f092 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:20:42.997193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-80730a62-a4f1-4f4a-9a78-9518cdd0f092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:43.158405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80730a62-a4f1-4f4a-9a78-9518cdd0f092 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:20:43.243432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:43.292419 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd03cf-595a-4487-8795-894d60f38555 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:20:43.293047 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd03cf-595a-4487-8795-894d60f38555 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:43.293322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd03cf-595a-4487-8795-894d60f38555 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:43.293613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd03cf-595a-4487-8795-894d60f38555 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:43.295269 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-86fd03cf-595a-4487-8795-894d60f38555 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:43.394083 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd03cf-595a-4487-8795-894d60f38555 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:20:43.395820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd03cf-595a-4487-8795-894d60f38555 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:20:43.416420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 674/1360] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:20:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:20:45.674963 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf5cef-23ba-4b72-a33f-d0dc15ec46a4 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:45.675481 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf5cef-23ba-4b72-a33f-d0dc15ec46a4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:45.675481 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf5cef-23ba-4b72-a33f-d0dc15ec46a4 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:45.676998 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 687/1361] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:20:45.682107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1ad6e-2644-4f25-80a5-41ae1092f48e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:45.682527 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1ad6e-2644-4f25-80a5-41ae1092f48e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:45.682666 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1ad6e-2644-4f25-80a5-41ae1092f48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:45.750801 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6ed1ad6e-2644-4f25-80a5-41ae1092f48e None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:45.752014 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6ed1ad6e-2644-4f25-80a5-41ae1092f48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:45.790218 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:45.859323 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 675/1362] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:20:45.904239 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf5cef-23ba-4b72-a33f-d0dc15ec46a4 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:20:45.906897 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ffcf5cef-23ba-4b72-a33f-d0dc15ec46a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:45.991991 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:46.082736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf5cef-23ba-4b72-a33f-d0dc15ec46a4 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:20:46.235198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1810179-fdcc-4619-90bc-835fd625bd57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:20:46.236179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1810179-fdcc-4619-90bc-835fd625bd57 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:46.236466 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1810179-fdcc-4619-90bc-835fd625bd57 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:46.236792 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1810179-fdcc-4619-90bc-835fd625bd57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:46.238496 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d1810179-fdcc-4619-90bc-835fd625bd57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:46.263163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:46.335545 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1810179-fdcc-4619-90bc-835fd625bd57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:20:46.337138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1810179-fdcc-4619-90bc-835fd625bd57 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:20:46.356804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 688/1363] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:20:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:20:48.707312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e78e78-09db-4bc0-8140-831a73d68049 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:48.707882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e78e78-09db-4bc0-8140-831a73d68049 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:48.707882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e78e78-09db-4bc0-8140-831a73d68049 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:48.709231 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 676/1364] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:20:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:20:48.714679 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba8e96-2e5a-4321-b379-3fae5a206524 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:48.715130 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba8e96-2e5a-4321-b379-3fae5a206524 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:48.715245 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba8e96-2e5a-4321-b379-3fae5a206524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:48.801722 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b4ba8e96-2e5a-4321-b379-3fae5a206524 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:48.802959 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b4ba8e96-2e5a-4321-b379-3fae5a206524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:48.913492 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:48.916507 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 689/1365] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:20:48.964697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c1f70-d28b-44a2-afbc-4e24155a723f None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:48.965221 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c1f70-d28b-44a2-afbc-4e24155a723f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:48.965622 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c1f70-d28b-44a2-afbc-4e24155a723f None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:48.967302 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 677/1366] 10.222.0.39 () {54 vars in 962 bytes} [Thu Jul 28 22:20:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:48.973059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0d9f2-87d8-484d-af73-f0cf6a9b2dd9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:48.973353 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0d9f2-87d8-484d-af73-f0cf6a9b2dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:48.973661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0d9f2-87d8-484d-af73-f0cf6a9b2dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:49.053821 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f0c0d9f2-87d8-484d-af73-f0cf6a9b2dd9 None None] MFA Rules not processed for user `77efe8046ea0468cb0c7c58c24fc4e75`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:49.055046 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f0c0d9f2-87d8-484d-af73-f0cf6a9b2dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:49.188161 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 690/1367] 10.222.0.39 () {58 vars in 1059 bytes} [Thu Jul 28 22:20:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:20:49.195904 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c1f70-d28b-44a2-afbc-4e24155a723f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:20:49.197913 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ff9c1f70-d28b-44a2-afbc-4e24155a723f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:49.368741 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c1f70-d28b-44a2-afbc-4e24155a723f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:20:49.390779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:49.492420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48cb1e7-f884-4310-9dad-17aa5dad7cfa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:20:49.493197 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cb1e7-f884-4310-9dad-17aa5dad7cfa service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:49.493609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cb1e7-f884-4310-9dad-17aa5dad7cfa service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:49.493974 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48cb1e7-f884-4310-9dad-17aa5dad7cfa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:49.495696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f48cb1e7-f884-4310-9dad-17aa5dad7cfa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:49.590275 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48cb1e7-f884-4310-9dad-17aa5dad7cfa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:20:49.591961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48cb1e7-f884-4310-9dad-17aa5dad7cfa service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:20:49.612525 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 678/1368] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:20:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:20:50.680956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd2122-776a-4863-8d34-b8b8f05d344d None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:50.681482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd2122-776a-4863-8d34-b8b8f05d344d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:50.681482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cd2122-776a-4863-8d34-b8b8f05d344d None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:50.683019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 691/1369] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:50.688193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65419e8d-4801-4ad1-8694-875041929452 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:50.688614 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65419e8d-4801-4ad1-8694-875041929452 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:50.688801 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65419e8d-4801-4ad1-8694-875041929452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:50.722071 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:50.760155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-65419e8d-4801-4ad1-8694-875041929452 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:50.761382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-65419e8d-4801-4ad1-8694-875041929452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:50.873612 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 679/1370] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:20:51.230918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1c42c-12b1-4ab9-9cc6-0278fbf1dafe None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:51.231499 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1c42c-12b1-4ab9-9cc6-0278fbf1dafe None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:51.231939 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1c42c-12b1-4ab9-9cc6-0278fbf1dafe None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:51.234550 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 692/1371] 10.222.0.39 () {54 vars in 964 bytes} [Thu Jul 28 22:20:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:51.239769 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596ba91-5211-4015-b4de-e0635c2e08b9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:51.240190 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596ba91-5211-4015-b4de-e0635c2e08b9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:51.240569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596ba91-5211-4015-b4de-e0635c2e08b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:51.321354 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a596ba91-5211-4015-b4de-e0635c2e08b9 None None] MFA Rules not processed for user `b4aa320b18ce43c3acfece4546fea0f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:51.322742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a596ba91-5211-4015-b4de-e0635c2e08b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:51.373334 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:51.407323 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 680/1372] 10.222.0.39 () {58 vars in 1061 bytes} [Thu Jul 28 22:20:51 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:20:51.414537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1c42c-12b1-4ab9-9cc6-0278fbf1dafe None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:20:51.416082 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-75d1c42c-12b1-4ab9-9cc6-0278fbf1dafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:51.472542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:51.529665 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1c42c-12b1-4ab9-9cc6-0278fbf1dafe None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:20:51.610476 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14585b-d153-4baa-8cba-30b23de36451 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:20:51.611103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14585b-d153-4baa-8cba-30b23de36451 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:51.611388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14585b-d153-4baa-8cba-30b23de36451 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:51.611666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14585b-d153-4baa-8cba-30b23de36451 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:51.613294 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6d14585b-d153-4baa-8cba-30b23de36451 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:51.706086 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a0012-7617-4800-b83c-fc98587ff2ae None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:51.707092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a0012-7617-4800-b83c-fc98587ff2ae None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:51.707474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a0012-7617-4800-b83c-fc98587ff2ae None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:51.709163 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 681/1373] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:20:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:51.714679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6188d-21f0-40ae-af9f-7ae56b2b6626 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:51.714986 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6188d-21f0-40ae-af9f-7ae56b2b6626 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:51.715252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6188d-21f0-40ae-af9f-7ae56b2b6626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:51.722775 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:51.738375 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14585b-d153-4baa-8cba-30b23de36451 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:20:51.739961 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14585b-d153-4baa-8cba-30b23de36451 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:20:51.759668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 693/1374] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:20:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:20:51.790082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f1c6188d-21f0-40ae-af9f-7ae56b2b6626 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:51.791268 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f1c6188d-21f0-40ae-af9f-7ae56b2b6626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:51.962734 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 682/1375] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:20:52.007201 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14585b-d153-4baa-8cba-30b23de36451 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:20:52.008410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6d14585b-d153-4baa-8cba-30b23de36451 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:52.110104 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14585b-d153-4baa-8cba-30b23de36451 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:20:52.212863 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:52.224943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff934fa6-dfdb-4699-a016-450f79b47b75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:20:52.225589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff934fa6-dfdb-4699-a016-450f79b47b75 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:52.225874 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff934fa6-dfdb-4699-a016-450f79b47b75 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:52.226148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff934fa6-dfdb-4699-a016-450f79b47b75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:52.227816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ff934fa6-dfdb-4699-a016-450f79b47b75 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:52.317845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff934fa6-dfdb-4699-a016-450f79b47b75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:20:52.319446 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff934fa6-dfdb-4699-a016-450f79b47b75 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:20:52.337748 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 694/1376] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:20:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:20:54.397907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9558f-d06d-465d-976c-c656a219b374 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:54.398398 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9558f-d06d-465d-976c-c656a219b374 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:54.398398 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9558f-d06d-465d-976c-c656a219b374 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:54.399767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 683/1377] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:54.405938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab98b2-2a76-4bfd-a3a7-479da2b56efb None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:54.406359 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab98b2-2a76-4bfd-a3a7-479da2b56efb None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:54.406463 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab98b2-2a76-4bfd-a3a7-479da2b56efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:54.472148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-76ab98b2-2a76-4bfd-a3a7-479da2b56efb None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:54.473366 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-76ab98b2-2a76-4bfd-a3a7-479da2b56efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:54.537346 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:54.577278 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 695/1378] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:20:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:20:54.621675 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9558f-d06d-465d-976c-c656a219b374 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:20:54.623177 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6ba9558f-d06d-465d-976c-c656a219b374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:54.675220 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:54.797406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9558f-d06d-465d-976c-c656a219b374 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:20:54.916318 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117fed77-7576-41a7-95d0-1e6dea0a3eca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:20:54.916936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fed77-7576-41a7-95d0-1e6dea0a3eca service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:54.917221 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fed77-7576-41a7-95d0-1e6dea0a3eca service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:54.917495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117fed77-7576-41a7-95d0-1e6dea0a3eca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:54.919134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-117fed77-7576-41a7-95d0-1e6dea0a3eca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:54.931383 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:55.011672 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117fed77-7576-41a7-95d0-1e6dea0a3eca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:20:55.013571 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117fed77-7576-41a7-95d0-1e6dea0a3eca service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:20:55.038557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 684/1379] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:20:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:20:57.122230 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c14e6-ba8c-4edc-8c31-8f6facf1fb03 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:57.122724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c14e6-ba8c-4edc-8c31-8f6facf1fb03 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:57.122809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c14e6-ba8c-4edc-8c31-8f6facf1fb03 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:57.124845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 696/1380] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:20:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:57.129920 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b82e5-a974-462d-bae2-ee24d50666e3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:57.130329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b82e5-a974-462d-bae2-ee24d50666e3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:57.130516 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b82e5-a974-462d-bae2-ee24d50666e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:57.209225 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ad2b82e5-a974-462d-bae2-ee24d50666e3 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:57.210533 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ad2b82e5-a974-462d-bae2-ee24d50666e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:57.307295 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:57.323224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 685/1381] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:20:57.367737 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c14e6-ba8c-4edc-8c31-8f6facf1fb03 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:20:57.371987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1a1c14e6-ba8c-4edc-8c31-8f6facf1fb03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:57.528024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c14e6-ba8c-4edc-8c31-8f6facf1fb03 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:20:57.579728 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:20:57.653926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65719da1-c099-439a-92cb-9da2851e2d8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:20:57.654650 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65719da1-c099-439a-92cb-9da2851e2d8a service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:57.654938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65719da1-c099-439a-92cb-9da2851e2d8a service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:57.655218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65719da1-c099-439a-92cb-9da2851e2d8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:57.656849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-65719da1-c099-439a-92cb-9da2851e2d8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:57.754085 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65719da1-c099-439a-92cb-9da2851e2d8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:20:57.755748 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65719da1-c099-439a-92cb-9da2851e2d8a service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:20:57.775305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 697/1382] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:20:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:20:59.870991 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd29b94-ab9a-4d5c-9dba-06687e84b748 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:59.871559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd29b94-ab9a-4d5c-9dba-06687e84b748 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:59.871559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd29b94-ab9a-4d5c-9dba-06687e84b748 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:59.872896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 686/1383] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:20:59.878203 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da39bb-afe5-4aee-a84e-aad649a9b68c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:20:59.878613 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da39bb-afe5-4aee-a84e-aad649a9b68c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:20:59.878759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da39bb-afe5-4aee-a84e-aad649a9b68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:20:59.950917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-02da39bb-afe5-4aee-a84e-aad649a9b68c None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:20:59.952769 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-02da39bb-afe5-4aee-a84e-aad649a9b68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:20:59.954980 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:00.054492 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 698/1384] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:20:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:00.098607 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd29b94-ab9a-4d5c-9dba-06687e84b748 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:00.100108 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0dd29b94-ab9a-4d5c-9dba-06687e84b748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:00.254574 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd29b94-ab9a-4d5c-9dba-06687e84b748 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:00.324223 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:00.396999 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a0aa2-223b-4496-b53c-ddc1c582bdd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:00.397762 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a0aa2-223b-4496-b53c-ddc1c582bdd5 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:00.398043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a0aa2-223b-4496-b53c-ddc1c582bdd5 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:00.398315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a0aa2-223b-4496-b53c-ddc1c582bdd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:00.399983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-804a0aa2-223b-4496-b53c-ddc1c582bdd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:00.491387 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a0aa2-223b-4496-b53c-ddc1c582bdd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:00.492913 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a0aa2-223b-4496-b53c-ddc1c582bdd5 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:00.511221 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 687/1385] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:21:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:21:02.540404 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3195e0-727d-4b60-ab26-4ab9823892a4 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:02.541008 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3195e0-727d-4b60-ab26-4ab9823892a4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:02.541197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3195e0-727d-4b60-ab26-4ab9823892a4 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:02.543137 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 699/1386] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:21:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:02.549749 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e40550-460d-455f-bae5-bb29f354fac2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:02.550182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e40550-460d-455f-bae5-bb29f354fac2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:02.550367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e40550-460d-455f-bae5-bb29f354fac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:02.616842 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-82e40550-460d-455f-bae5-bb29f354fac2 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:02.618065 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-82e40550-460d-455f-bae5-bb29f354fac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:02.654015 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:02.743688 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 688/1387] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:21:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:02.798521 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3195e0-727d-4b60-ab26-4ab9823892a4 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:02.800161 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-dc3195e0-727d-4b60-ab26-4ab9823892a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:02.971021 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:02.995992 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3195e0-727d-4b60-ab26-4ab9823892a4 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:03.108577 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638843f-4923-4694-8da0-0196b8b1df59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:03.109197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638843f-4923-4694-8da0-0196b8b1df59 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:03.109706 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638843f-4923-4694-8da0-0196b8b1df59 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:03.109834 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638843f-4923-4694-8da0-0196b8b1df59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:03.111373 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2638843f-4923-4694-8da0-0196b8b1df59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:03.213148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638843f-4923-4694-8da0-0196b8b1df59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:03.215588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638843f-4923-4694-8da0-0196b8b1df59 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:03.222274 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:03.236993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 700/1388] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:21:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:21:05.426216 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47374924-6832-4199-805a-22ac0b5a198d None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:05.426734 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47374924-6832-4199-805a-22ac0b5a198d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:05.426734 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47374924-6832-4199-805a-22ac0b5a198d None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:05.428033 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 689/1389] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:05.433147 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a722926-6ac4-4f83-83aa-fda192c59a34 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:05.433581 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a722926-6ac4-4f83-83aa-fda192c59a34 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:05.433764 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a722926-6ac4-4f83-83aa-fda192c59a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:05.498882 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3a722926-6ac4-4f83-83aa-fda192c59a34 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:05.500180 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3a722926-6ac4-4f83-83aa-fda192c59a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:05.604412 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 701/1390] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:05.651049 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47374924-6832-4199-805a-22ac0b5a198d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:05.652652 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-47374924-6832-4199-805a-22ac0b5a198d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:05.815938 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47374924-6832-4199-805a-22ac0b5a198d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:05.824771 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:05.947868 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0cc7c-fa24-4346-abd1-183532cb9586 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:05.948484 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0cc7c-fa24-4346-abd1-183532cb9586 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:05.948764 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0cc7c-fa24-4346-abd1-183532cb9586 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:05.949062 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0cc7c-fa24-4346-abd1-183532cb9586 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:05.950909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d1a0cc7c-fa24-4346-abd1-183532cb9586 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:06.069380 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0cc7c-fa24-4346-abd1-183532cb9586 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:06.070925 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0cc7c-fa24-4346-abd1-183532cb9586 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:06.092229 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 690/1391] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:21:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:21:06.963427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6b7aa-7a6f-4eca-9ef7-66443a7fc916 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:06.963994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6b7aa-7a6f-4eca-9ef7-66443a7fc916 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:06.964090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6b7aa-7a6f-4eca-9ef7-66443a7fc916 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:06.965491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 702/1392] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:06.971354 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ffa0c-66c7-4cf1-8989-fe5363a3fcea None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:06.971838 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ffa0c-66c7-4cf1-8989-fe5363a3fcea None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:06.971938 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ffa0c-66c7-4cf1-8989-fe5363a3fcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:06.987866 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:07.038207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-333ffa0c-66c7-4cf1-8989-fe5363a3fcea None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:07.039420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-333ffa0c-66c7-4cf1-8989-fe5363a3fcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:07.137491 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 691/1393] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:21:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:07.194261 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2a7b8-446f-4106-bc21-0fbd85421a56 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:07.194553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2a7b8-446f-4106-bc21-0fbd85421a56 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:07.194830 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2a7b8-446f-4106-bc21-0fbd85421a56 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:07.196195 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 703/1394] 10.222.0.39 () {54 vars in 966 bytes} [Thu Jul 28 22:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:07.201704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97a617-4dd5-4f88-9cfd-8a68b7b55700 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:07.201999 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97a617-4dd5-4f88-9cfd-8a68b7b55700 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:07.202280 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97a617-4dd5-4f88-9cfd-8a68b7b55700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:07.283792 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7a97a617-4dd5-4f88-9cfd-8a68b7b55700 None None] MFA Rules not processed for user `0fecb440c0874ede9f3cf544cfed93ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:07.284928 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7a97a617-4dd5-4f88-9cfd-8a68b7b55700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:07.320577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:07.364601 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 692/1395] 10.222.0.39 () {58 vars in 1063 bytes} [Thu Jul 28 22:21:07 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:07.372175 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2a7b8-446f-4106-bc21-0fbd85421a56 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:07.374258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-43b2a7b8-446f-4106-bc21-0fbd85421a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:07.449218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:07.481658 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2a7b8-446f-4106-bc21-0fbd85421a56 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:07.556338 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae254829-21fe-4ec6-a875-bb704ba1b1b8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fecb440c0874ede9f3cf544cfed93ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:07.556873 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae254829-21fe-4ec6-a875-bb704ba1b1b8 service glance] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:07.557153 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae254829-21fe-4ec6-a875-bb704ba1b1b8 service glance] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:07.557446 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae254829-21fe-4ec6-a875-bb704ba1b1b8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:07.559074 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ae254829-21fe-4ec6-a875-bb704ba1b1b8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:07.667522 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae254829-21fe-4ec6-a875-bb704ba1b1b8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:07.668956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae254829-21fe-4ec6-a875-bb704ba1b1b8 service glance] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:07.684245 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:07.689411 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 704/1396] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:21:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:21:08.195496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05b0ea-0a11-4958-aad7-3c43d4dc2210 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:08.195892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05b0ea-0a11-4958-aad7-3c43d4dc2210 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:08.196055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05b0ea-0a11-4958-aad7-3c43d4dc2210 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:08.197405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 693/1397] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:08.202840 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2d121-4a2c-42f8-8808-378939478c0b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:08.203256 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2d121-4a2c-42f8-8808-378939478c0b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:08.203416 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2d121-4a2c-42f8-8808-378939478c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:08.273328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-46d2d121-4a2c-42f8-8808-378939478c0b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:08.274617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-46d2d121-4a2c-42f8-8808-378939478c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:08.383713 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 705/1398] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:21:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:08.431553 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05b0ea-0a11-4958-aad7-3c43d4dc2210 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:08.433074 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8a05b0ea-0a11-4958-aad7-3c43d4dc2210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:08.444699 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4b5fe-94f5-453a-84a1-bfcaf91e3ae3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:08.445024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4b5fe-94f5-453a-84a1-bfcaf91e3ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:08.445355 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4b5fe-94f5-453a-84a1-bfcaf91e3ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:08.513193 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f9c4b5fe-94f5-453a-84a1-bfcaf91e3ae3 None None] MFA Rules not processed for user `85626350279c40ea99da93b9ed6994b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:08.514871 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f9c4b5fe-94f5-453a-84a1-bfcaf91e3ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:08.544802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:08.581670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05b0ea-0a11-4958-aad7-3c43d4dc2210 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:08.606087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 706/1399] 10.222.0.39 () {58 vars in 1009 bytes} [Thu Jul 28 22:21:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:21:08.622670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:08.649186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77aecc-632b-4ebe-90ee-839a19637c4c None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:08.649574 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77aecc-632b-4ebe-90ee-839a19637c4c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:08.649862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c77aecc-632b-4ebe-90ee-839a19637c4c None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:08.651227 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 707/1400] 10.222.0.39 () {54 vars in 943 bytes} [Thu Jul 28 22:21:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:21:08.657358 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb196a-717f-488e-a197-36abc0605209 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:08.657681 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb196a-717f-488e-a197-36abc0605209 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:08.657959 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb196a-717f-488e-a197-36abc0605209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:08.702946 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0dc55-ef8e-4fcf-acfd-6fcf50baf14e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:08.704083 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0dc55-ef8e-4fcf-acfd-6fcf50baf14e service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:08.704368 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0dc55-ef8e-4fcf-acfd-6fcf50baf14e service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:08.704643 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f0dc55-ef8e-4fcf-acfd-6fcf50baf14e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:08.706344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-13f0dc55-ef8e-4fcf-acfd-6fcf50baf14e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:08.739397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8efb196a-717f-488e-a197-36abc0605209 None None] MFA Rules not processed for user `e3c9107872f147f6ab8698b67bb545af`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:08.740634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8efb196a-717f-488e-a197-36abc0605209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:08.799983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0dc55-ef8e-4fcf-acfd-6fcf50baf14e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:08.801633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f0dc55-ef8e-4fcf-acfd-6fcf50baf14e service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:08.821733 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 694/1401] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:21:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:21:08.855744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 708/1402] 10.222.0.39 () {58 vars in 1040 bytes} [Thu Jul 28 22:21:08 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:21:08.865665 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0dc55-ef8e-4fcf-acfd-6fcf50baf14e service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:08.866866 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-13f0dc55-ef8e-4fcf-acfd-6fcf50baf14e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:08.923028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:08.972820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f0dc55-ef8e-4fcf-acfd-6fcf50baf14e service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:09.084059 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69da5e35-d4eb-4ede-8345-32b89a225402 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9107872f147f6ab8698b67bb545af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:09.084672 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da5e35-d4eb-4ede-8345-32b89a225402 service swift] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:09.084949 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da5e35-d4eb-4ede-8345-32b89a225402 service swift] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:09.085216 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69da5e35-d4eb-4ede-8345-32b89a225402 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:09.086907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-69da5e35-d4eb-4ede-8345-32b89a225402 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:09.173699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:09.178077 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da5e35-d4eb-4ede-8345-32b89a225402 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:09.179773 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69da5e35-d4eb-4ede-8345-32b89a225402 service swift] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:09.188742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 695/1403] 10.222.0.39 () {60 vars in 1384 bytes} [Thu Jul 28 22:21:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 22:21:10.905703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362283f8-6cab-474b-ad4c-edcbae7401a1 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:10.905703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362283f8-6cab-474b-ad4c-edcbae7401a1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:10.905703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362283f8-6cab-474b-ad4c-edcbae7401a1 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:10.908294 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 709/1404] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:21:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:10.913058 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79d571-a1c6-4269-a90a-320bf04250a3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:10.913951 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79d571-a1c6-4269-a90a-320bf04250a3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:10.914310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79d571-a1c6-4269-a90a-320bf04250a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:10.990304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8b79d571-a1c6-4269-a90a-320bf04250a3 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:10.991596 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8b79d571-a1c6-4269-a90a-320bf04250a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:11.099798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 696/1405] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:21:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:11.149542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362283f8-6cab-474b-ad4c-edcbae7401a1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:11.151248 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-362283f8-6cab-474b-ad4c-edcbae7401a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:11.154493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:11.321277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362283f8-6cab-474b-ad4c-edcbae7401a1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:11.402782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:11.453342 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f006c7f-d84f-451f-9be0-0efacd49c297 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:11.454005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f006c7f-d84f-451f-9be0-0efacd49c297 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:11.454289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f006c7f-d84f-451f-9be0-0efacd49c297 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:11.454561 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f006c7f-d84f-451f-9be0-0efacd49c297 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:11.456270 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1f006c7f-d84f-451f-9be0-0efacd49c297 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:11.558670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f006c7f-d84f-451f-9be0-0efacd49c297 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:11.560580 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f006c7f-d84f-451f-9be0-0efacd49c297 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:11.580281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 710/1406] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:21:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:21:13.636471 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b52ac-c12e-4d77-9d2a-c946a9628c12 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:13.637296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b52ac-c12e-4d77-9d2a-c946a9628c12 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:13.637296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b52ac-c12e-4d77-9d2a-c946a9628c12 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:13.638969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 697/1407] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:21:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:13.644111 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37545b55-f8ac-486d-81f9-789fe6bba9f5 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:13.644559 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37545b55-f8ac-486d-81f9-789fe6bba9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:13.644653 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37545b55-f8ac-486d-81f9-789fe6bba9f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:13.726196 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-37545b55-f8ac-486d-81f9-789fe6bba9f5 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:13.727500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-37545b55-f8ac-486d-81f9-789fe6bba9f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:13.768569 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:13.837023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 711/1408] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:21:13.886005 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b52ac-c12e-4d77-9d2a-c946a9628c12 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:13.887970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-787b52ac-c12e-4d77-9d2a-c946a9628c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:13.976337 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:14.057540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b52ac-c12e-4d77-9d2a-c946a9628c12 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:14.198109 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1986f9e5-7e33-4b22-a6cc-fb342bfebd97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:14.198735 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986f9e5-7e33-4b22-a6cc-fb342bfebd97 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:14.199017 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986f9e5-7e33-4b22-a6cc-fb342bfebd97 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:14.199292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1986f9e5-7e33-4b22-a6cc-fb342bfebd97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:14.201668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1986f9e5-7e33-4b22-a6cc-fb342bfebd97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:14.254908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:14.319644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986f9e5-7e33-4b22-a6cc-fb342bfebd97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:14.321588 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1986f9e5-7e33-4b22-a6cc-fb342bfebd97 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:14.342281 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 698/1409] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:21:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:21:23.834950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=26 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:24.339560 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=30 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:31.406025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970075cf-e9ec-4377-8494-b7dc7abfeb4e None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:31.406561 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970075cf-e9ec-4377-8494-b7dc7abfeb4e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:31.406561 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970075cf-e9ec-4377-8494-b7dc7abfeb4e None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:31.407861 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 712/1410] 10.222.0.39 () {54 vars in 918 bytes} [Thu Jul 28 22:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:31.412918 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893e8ec-123c-4a65-9d31-e12baacff724 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:31.413347 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893e8ec-123c-4a65-9d31-e12baacff724 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:31.413490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893e8ec-123c-4a65-9d31-e12baacff724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:31.489340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7893e8ec-123c-4a65-9d31-e12baacff724 None None] MFA Rules not processed for user `fe003cd7bfbe4b92b1d64d300c3e7d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:31.490598 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7893e8ec-123c-4a65-9d31-e12baacff724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:31.594637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 699/1411] 10.222.0.39 () {58 vars in 1015 bytes} [Thu Jul 28 22:21:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:31.609846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d308d6-c478-4034-b985-0fb2d467691d None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:31.610135 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d308d6-c478-4034-b985-0fb2d467691d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:31.610413 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d308d6-c478-4034-b985-0fb2d467691d None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:31.611764 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 713/1412] 10.222.0.39 () {54 vars in 918 bytes} [Thu Jul 28 22:21:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:31.616288 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1f2f5-e1f3-459d-8d68-ccba974368c8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:31.616578 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1f2f5-e1f3-459d-8d68-ccba974368c8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:31.616843 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1f2f5-e1f3-459d-8d68-ccba974368c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:31.688237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:31.691335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-04a1f2f5-e1f3-459d-8d68-ccba974368c8 None None] MFA Rules not processed for user `77efe8046ea0468cb0c7c58c24fc4e75`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:31.693105 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-04a1f2f5-e1f3-459d-8d68-ccba974368c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:31.808861 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 700/1413] 10.222.0.39 () {58 vars in 1015 bytes} [Thu Jul 28 22:21:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:34.300146 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbe5eb-6cb1-4cb5-bde8-106d7f5bf9c1 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:34.300669 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbe5eb-6cb1-4cb5-bde8-106d7f5bf9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:34.300669 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fbe5eb-6cb1-4cb5-bde8-106d7f5bf9c1 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:34.302007 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 714/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 22:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:34.307303 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5784ad8-d34b-4b2e-a274-3f8f42391140 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:34.307751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5784ad8-d34b-4b2e-a274-3f8f42391140 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:34.307878 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5784ad8-d34b-4b2e-a274-3f8f42391140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:34.376624 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a5784ad8-d34b-4b2e-a274-3f8f42391140 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:34.377876 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a5784ad8-d34b-4b2e-a274-3f8f42391140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:34.432671 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:34.475385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 701/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 22:21:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:34.519361 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbe5eb-6cb1-4cb5-bde8-106d7f5bf9c1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:34.520898 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-36fbe5eb-6cb1-4cb5-bde8-106d7f5bf9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:34.663283 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fbe5eb-6cb1-4cb5-bde8-106d7f5bf9c1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:34.735663 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:34.776243 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f964a-3906-424d-9d25-4b6599f02a68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:34.776862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f964a-3906-424d-9d25-4b6599f02a68 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:34.777144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f964a-3906-424d-9d25-4b6599f02a68 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:34.777420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f964a-3906-424d-9d25-4b6599f02a68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:34.779103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1a3f964a-3906-424d-9d25-4b6599f02a68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:34.867390 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f964a-3906-424d-9d25-4b6599f02a68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:34.868989 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f964a-3906-424d-9d25-4b6599f02a68 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:34.887829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 715/1416] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:21:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:21:35.789419 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dac2fa-bf54-4e30-9e12-f8a5a404c8fe None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:35.790040 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dac2fa-bf54-4e30-9e12-f8a5a404c8fe None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:35.790040 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dac2fa-bf54-4e30-9e12-f8a5a404c8fe None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:35.791363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 702/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 22:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:35.796329 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa6f5c-bde5-48fb-8c0c-a0a39d5de087 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:35.796762 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa6f5c-bde5-48fb-8c0c-a0a39d5de087 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:35.796871 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa6f5c-bde5-48fb-8c0c-a0a39d5de087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:35.874672 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-dbaa6f5c-bde5-48fb-8c0c-a0a39d5de087 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:35.875890 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-dbaa6f5c-bde5-48fb-8c0c-a0a39d5de087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:35.903158 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:35.971703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 716/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 22:21:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:36.126069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dac2fa-bf54-4e30-9e12-f8a5a404c8fe None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:36.127562 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e3dac2fa-bf54-4e30-9e12-f8a5a404c8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:36.271513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dac2fa-bf54-4e30-9e12-f8a5a404c8fe None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:36.312119 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:36.383974 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225208a4-a6b1-4b2c-894d-54fcdf9a589c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:36.384509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225208a4-a6b1-4b2c-894d-54fcdf9a589c service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:36.384785 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225208a4-a6b1-4b2c-894d-54fcdf9a589c service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:36.385053 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225208a4-a6b1-4b2c-894d-54fcdf9a589c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:36.386690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-225208a4-a6b1-4b2c-894d-54fcdf9a589c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:36.474641 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225208a4-a6b1-4b2c-894d-54fcdf9a589c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:36.476220 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225208a4-a6b1-4b2c-894d-54fcdf9a589c service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:36.494585 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 703/1419] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:21:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:21:37.392595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ceaf42-98f3-4fcf-9622-1d2134b36e15 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:37.393159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ceaf42-98f3-4fcf-9622-1d2134b36e15 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:37.393159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ceaf42-98f3-4fcf-9622-1d2134b36e15 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:37.394564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 717/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 22:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:37.399489 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83a9cd-64b1-4b7a-834b-94425af3ec33 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:37.399931 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83a9cd-64b1-4b7a-834b-94425af3ec33 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:37.400069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83a9cd-64b1-4b7a-834b-94425af3ec33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:37.454829 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:37.476932 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ce83a9cd-64b1-4b7a-834b-94425af3ec33 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:37.478161 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ce83a9cd-64b1-4b7a-834b-94425af3ec33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:37.573932 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 704/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 22:21:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:37.730000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ceaf42-98f3-4fcf-9622-1d2134b36e15 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:37.731491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-70ceaf42-98f3-4fcf-9622-1d2134b36e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:37.876144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ceaf42-98f3-4fcf-9622-1d2134b36e15 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:37.888993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:37.990240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3941f0e-e629-43b4-9635-bc52f878337e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:37.990779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3941f0e-e629-43b4-9635-bc52f878337e service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:37.991061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3941f0e-e629-43b4-9635-bc52f878337e service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:37.991337 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3941f0e-e629-43b4-9635-bc52f878337e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:37.992964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d3941f0e-e629-43b4-9635-bc52f878337e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:38.080691 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3941f0e-e629-43b4-9635-bc52f878337e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:38.082588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3941f0e-e629-43b4-9635-bc52f878337e service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:38.101466 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 718/1422] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:21:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:21:38.962744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132fa17-705d-4c6c-8378-c0f8d90d550d None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:38.963288 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132fa17-705d-4c6c-8378-c0f8d90d550d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:38.963288 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132fa17-705d-4c6c-8378-c0f8d90d550d None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:38.964656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 705/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 22:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:38.970064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96ba18-93ac-44d2-850b-ea3cb0bbca97 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:38.970492 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96ba18-93ac-44d2-850b-ea3cb0bbca97 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:38.970618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f96ba18-93ac-44d2-850b-ea3cb0bbca97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:38.990502 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:39.047883 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6f96ba18-93ac-44d2-850b-ea3cb0bbca97 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:39.049090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6f96ba18-93ac-44d2-850b-ea3cb0bbca97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:39.142475 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 719/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 22:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:39.297920 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132fa17-705d-4c6c-8378-c0f8d90d550d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:39.299424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3132fa17-705d-4c6c-8378-c0f8d90d550d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:39.411674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:39.447339 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3132fa17-705d-4c6c-8378-c0f8d90d550d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:39.559018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea753c-84fc-43f0-931c-0792ecd44fdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:39.559772 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea753c-84fc-43f0-931c-0792ecd44fdd service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:39.560092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea753c-84fc-43f0-931c-0792ecd44fdd service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:39.560364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea753c-84fc-43f0-931c-0792ecd44fdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:39.561997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-91ea753c-84fc-43f0-931c-0792ecd44fdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:39.638411 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:39.652365 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea753c-84fc-43f0-931c-0792ecd44fdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:39.654605 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea753c-84fc-43f0-931c-0792ecd44fdd service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:39.672486 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 706/1425] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:21:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:21:40.546895 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc417f43-75b8-47a7-aeae-a72e1361b0c9 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:40.547448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc417f43-75b8-47a7-aeae-a72e1361b0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:40.547448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc417f43-75b8-47a7-aeae-a72e1361b0c9 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:40.548736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 720/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 22:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:40.553729 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b3009-6a7f-4c34-b6a6-ae0a70a11745 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:40.554145 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b3009-6a7f-4c34-b6a6-ae0a70a11745 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:40.554275 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b3009-6a7f-4c34-b6a6-ae0a70a11745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:40.630081 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-bb8b3009-6a7f-4c34-b6a6-ae0a70a11745 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:40.631280 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bb8b3009-6a7f-4c34-b6a6-ae0a70a11745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:40.727084 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 707/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 22:21:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:40.878226 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc417f43-75b8-47a7-aeae-a72e1361b0c9 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:40.879748 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bc417f43-75b8-47a7-aeae-a72e1361b0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:40.963370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:41.024814 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc417f43-75b8-47a7-aeae-a72e1361b0c9 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:41.135674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920e9f20-aa91-4515-a804-6ab3e7e3875d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:41.136488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e9f20-aa91-4515-a804-6ab3e7e3875d service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:41.136771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e9f20-aa91-4515-a804-6ab3e7e3875d service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:41.137047 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920e9f20-aa91-4515-a804-6ab3e7e3875d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:41.139236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-920e9f20-aa91-4515-a804-6ab3e7e3875d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:41.187790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:41.230908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e9f20-aa91-4515-a804-6ab3e7e3875d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:41.233096 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920e9f20-aa91-4515-a804-6ab3e7e3875d service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:41.251124 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 721/1428] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:21:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:21:42.053827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0cec52-13d7-44e3-9752-48cec87a2b65 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:42.054487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0cec52-13d7-44e3-9752-48cec87a2b65 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:42.054487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0cec52-13d7-44e3-9752-48cec87a2b65 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:42.055771 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 708/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 22:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:42.061379 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb7f57-10cb-4779-9630-18f9acb5b4c7 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:42.061832 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb7f57-10cb-4779-9630-18f9acb5b4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:42.061977 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb7f57-10cb-4779-9630-18f9acb5b4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:42.140379 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c0cb7f57-10cb-4779-9630-18f9acb5b4c7 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:42.141677 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c0cb7f57-10cb-4779-9630-18f9acb5b4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:42.257442 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 722/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 22:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:42.407470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0cec52-13d7-44e3-9752-48cec87a2b65 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:42.409008 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ca0cec52-13d7-44e3-9752-48cec87a2b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:42.457404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:42.555435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0cec52-13d7-44e3-9752-48cec87a2b65 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:42.667535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978ffde-fda7-4a4f-90d3-2852f906e840 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:42.668068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978ffde-fda7-4a4f-90d3-2852f906e840 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:42.668346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978ffde-fda7-4a4f-90d3-2852f906e840 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:42.668614 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978ffde-fda7-4a4f-90d3-2852f906e840 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:42.670750 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5978ffde-fda7-4a4f-90d3-2852f906e840 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:42.677574 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:42.761668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5978ffde-fda7-4a4f-90d3-2852f906e840 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:42.763601 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5978ffde-fda7-4a4f-90d3-2852f906e840 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:42.781885 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 709/1431] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:21:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:21:43.591679 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b936e6-b8cc-454b-be2e-f60fcce31573 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:43.592215 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b936e6-b8cc-454b-be2e-f60fcce31573 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:43.592215 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b936e6-b8cc-454b-be2e-f60fcce31573 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:43.593553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 723/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 22:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:43.598583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58992331-fca6-4010-bc18-706e7ce3b2b5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:43.599018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58992331-fca6-4010-bc18-706e7ce3b2b5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:43.599146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58992331-fca6-4010-bc18-706e7ce3b2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:43.674440 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-58992331-fca6-4010-bc18-706e7ce3b2b5 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:43.675655 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-58992331-fca6-4010-bc18-706e7ce3b2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:43.742820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:43.772943 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 710/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 22:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:43.923574 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b936e6-b8cc-454b-be2e-f60fcce31573 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:43.925092 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-47b936e6-b8cc-454b-be2e-f60fcce31573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:43.943808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:44.073307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b936e6-b8cc-454b-be2e-f60fcce31573 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:44.163559 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:44.191605 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb89209-9eda-42d3-931d-adc16135ac46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:44.192224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb89209-9eda-42d3-931d-adc16135ac46 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:44.192504 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb89209-9eda-42d3-931d-adc16135ac46 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:44.192780 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb89209-9eda-42d3-931d-adc16135ac46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:44.194456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ffb89209-9eda-42d3-931d-adc16135ac46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:44.284438 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb89209-9eda-42d3-931d-adc16135ac46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:44.286055 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb89209-9eda-42d3-931d-adc16135ac46 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:44.304044 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 724/1434] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:21:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:21:45.117368 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602ff5b-22f7-4b2e-b903-ce9da8fb26e5 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:45.117958 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602ff5b-22f7-4b2e-b903-ce9da8fb26e5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:45.117958 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602ff5b-22f7-4b2e-b903-ce9da8fb26e5 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:45.119298 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 711/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 22:21:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:45.124841 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2458ea-9bc0-4d85-b458-a02d555ec302 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:45.125250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2458ea-9bc0-4d85-b458-a02d555ec302 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:45.125397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2458ea-9bc0-4d85-b458-a02d555ec302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:45.204356 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-bb2458ea-9bc0-4d85-b458-a02d555ec302 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:45.205591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bb2458ea-9bc0-4d85-b458-a02d555ec302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:45.249704 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:45.305358 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 725/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 22:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:45.461072 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602ff5b-22f7-4b2e-b903-ce9da8fb26e5 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:45.462588 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5602ff5b-22f7-4b2e-b903-ce9da8fb26e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:45.604983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602ff5b-22f7-4b2e-b903-ce9da8fb26e5 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:45.659778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:45.721235 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5617aa-4d47-463d-8dd9-6035c5edaf1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:45.721802 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5617aa-4d47-463d-8dd9-6035c5edaf1c service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:45.722086 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5617aa-4d47-463d-8dd9-6035c5edaf1c service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:45.722361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5617aa-4d47-463d-8dd9-6035c5edaf1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:45.723976 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8e5617aa-4d47-463d-8dd9-6035c5edaf1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:45.813543 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5617aa-4d47-463d-8dd9-6035c5edaf1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:45.815167 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5617aa-4d47-463d-8dd9-6035c5edaf1c service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:45.832678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 712/1437] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:21:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:21:46.637653 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940a97e-ce68-4296-be18-9decdd27df40 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:46.638208 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940a97e-ce68-4296-be18-9decdd27df40 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:46.638332 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940a97e-ce68-4296-be18-9decdd27df40 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:46.639625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 726/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 22:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:46.644583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a03e95-7638-4b77-81c1-d18608f3fbdc None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:46.645031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a03e95-7638-4b77-81c1-d18608f3fbdc None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:46.645131 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a03e95-7638-4b77-81c1-d18608f3fbdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:46.719916 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-06a03e95-7638-4b77-81c1-d18608f3fbdc None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:46.721121 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-06a03e95-7638-4b77-81c1-d18608f3fbdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:46.723148 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:46.819042 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 713/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 22:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:46.974409 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940a97e-ce68-4296-be18-9decdd27df40 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:46.975828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f940a97e-ce68-4296-be18-9decdd27df40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:47.123663 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f940a97e-ce68-4296-be18-9decdd27df40 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:47.150841 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:47.240292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd97916-bfc8-44cd-87eb-d8478ca5a8d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:47.240830 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd97916-bfc8-44cd-87eb-d8478ca5a8d9 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:47.241111 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd97916-bfc8-44cd-87eb-d8478ca5a8d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:47.241385 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd97916-bfc8-44cd-87eb-d8478ca5a8d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:47.243029 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1bd97916-bfc8-44cd-87eb-d8478ca5a8d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:47.334787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd97916-bfc8-44cd-87eb-d8478ca5a8d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:47.336383 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd97916-bfc8-44cd-87eb-d8478ca5a8d9 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:47.355020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 727/1440] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:21:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:21:49.247021 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e24b3-ebf2-4f82-aa42-f9167de9c3c4 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:49.247556 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e24b3-ebf2-4f82-aa42-f9167de9c3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:49.247556 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3e24b3-ebf2-4f82-aa42-f9167de9c3c4 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:49.248906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 714/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 22:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:49.254122 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf1487-e6df-487c-a035-16c14423ae64 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:49.254552 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf1487-e6df-487c-a035-16c14423ae64 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:49.254664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf1487-e6df-487c-a035-16c14423ae64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:49.296553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:49.321996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d4bf1487-e6df-487c-a035-16c14423ae64 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:49.323224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d4bf1487-e6df-487c-a035-16c14423ae64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:49.419051 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 728/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 22:21:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:21:49.458400 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e24b3-ebf2-4f82-aa42-f9167de9c3c4 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:49.459923 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9e3e24b3-ebf2-4f82-aa42-f9167de9c3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:49.566580 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3e24b3-ebf2-4f82-aa42-f9167de9c3c4 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:49.598140 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:49.640153 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a43167-d27a-4244-b534-fcfa11ddfa95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:49.640767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a43167-d27a-4244-b534-fcfa11ddfa95 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:49.641047 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a43167-d27a-4244-b534-fcfa11ddfa95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:49.641321 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a43167-d27a-4244-b534-fcfa11ddfa95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:49.642974 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-76a43167-d27a-4244-b534-fcfa11ddfa95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:49.750326 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a43167-d27a-4244-b534-fcfa11ddfa95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:49.751682 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a43167-d27a-4244-b534-fcfa11ddfa95 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:49.770382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 715/1443] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:21:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:21:51.984207 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbaeeac-0d58-4380-af45-65f03714c363 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:51.984776 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbaeeac-0d58-4380-af45-65f03714c363 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:51.984776 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbaeeac-0d58-4380-af45-65f03714c363 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:51.986177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 729/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 22:21:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:51.991405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17937084-54f0-4d82-af3b-0ca8b995a003 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:51.991801 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17937084-54f0-4d82-af3b-0ca8b995a003 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:51.991962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17937084-54f0-4d82-af3b-0ca8b995a003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:52.058676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-17937084-54f0-4d82-af3b-0ca8b995a003 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:52.060423 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-17937084-54f0-4d82-af3b-0ca8b995a003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:52.073053 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:52.159634 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 716/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 22:21:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:52.197543 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbaeeac-0d58-4380-af45-65f03714c363 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:52.199090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-abbaeeac-0d58-4380-af45-65f03714c363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:52.301786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbaeeac-0d58-4380-af45-65f03714c363 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:52.314299 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:52.375911 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41e58f-3169-4bf7-8dcf-fb22deb05a2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:52.376526 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41e58f-3169-4bf7-8dcf-fb22deb05a2d service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:52.376811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41e58f-3169-4bf7-8dcf-fb22deb05a2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:52.377088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41e58f-3169-4bf7-8dcf-fb22deb05a2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:52.378736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1e41e58f-3169-4bf7-8dcf-fb22deb05a2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:52.486667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41e58f-3169-4bf7-8dcf-fb22deb05a2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:52.488090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41e58f-3169-4bf7-8dcf-fb22deb05a2d service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:52.506597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 730/1446] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:21:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:21:54.806949 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28ac55-c0f5-4b33-a27d-d590daff0dd1 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:54.807551 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28ac55-c0f5-4b33-a27d-d590daff0dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:54.807551 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28ac55-c0f5-4b33-a27d-d590daff0dd1 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:54.808959 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 717/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 22:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:54.814308 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56916bd-8bdc-4766-9d1b-46fb28c1a9f3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:54.814748 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56916bd-8bdc-4766-9d1b-46fb28c1a9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:54.814865 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56916bd-8bdc-4766-9d1b-46fb28c1a9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:54.863081 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:54.880977 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f56916bd-8bdc-4766-9d1b-46fb28c1a9f3 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:54.882242 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f56916bd-8bdc-4766-9d1b-46fb28c1a9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:54.980243 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 731/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 22:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:55.021191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28ac55-c0f5-4b33-a27d-d590daff0dd1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:55.022748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3d28ac55-c0f5-4b33-a27d-d590daff0dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:55.127623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d28ac55-c0f5-4b33-a27d-d590daff0dd1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:55.168739 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:55.202413 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf25c323-dda9-4ae1-851c-9a5b6374acab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:55.203030 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25c323-dda9-4ae1-851c-9a5b6374acab service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:55.203306 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25c323-dda9-4ae1-851c-9a5b6374acab service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:55.203576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf25c323-dda9-4ae1-851c-9a5b6374acab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:55.205293 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bf25c323-dda9-4ae1-851c-9a5b6374acab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:55.313232 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf25c323-dda9-4ae1-851c-9a5b6374acab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:55.314695 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf25c323-dda9-4ae1-851c-9a5b6374acab service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:55.334297 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 718/1449] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:21:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:21:57.439561 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70436e-84a7-4510-8ea3-28c2575cbbd8 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:57.440087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70436e-84a7-4510-8ea3-28c2575cbbd8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:57.440087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70436e-84a7-4510-8ea3-28c2575cbbd8 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:57.441439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 732/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 22:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:21:57.446575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a631fa4-bbeb-4389-9b01-ccd27e30e2cf None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:57.447004 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a631fa4-bbeb-4389-9b01-ccd27e30e2cf None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:57.447135 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a631fa4-bbeb-4389-9b01-ccd27e30e2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:57.514827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4a631fa4-bbeb-4389-9b01-ccd27e30e2cf None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:21:57.516033 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4a631fa4-bbeb-4389-9b01-ccd27e30e2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:57.537007 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:57.618618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 719/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 22:21:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:21:57.658600 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70436e-84a7-4510-8ea3-28c2575cbbd8 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:21:57.660111 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bf70436e-84a7-4510-8ea3-28c2575cbbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:57.760478 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf70436e-84a7-4510-8ea3-28c2575cbbd8 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:21:57.780756 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:21:57.833948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89e7cb-6183-47f8-aff1-91dacf520996 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:21:57.834562 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89e7cb-6183-47f8-aff1-91dacf520996 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:21:57.834845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89e7cb-6183-47f8-aff1-91dacf520996 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:21:57.835119 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89e7cb-6183-47f8-aff1-91dacf520996 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:21:57.836831 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9a89e7cb-6183-47f8-aff1-91dacf520996 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:21:57.947282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a89e7cb-6183-47f8-aff1-91dacf520996 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:21:57.948719 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a89e7cb-6183-47f8-aff1-91dacf520996 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:21:57.967727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 733/1452] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:21:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:22:00.055974 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b549fc3-acd7-4561-9887-1b956c8e6ea1 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:00.056542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b549fc3-acd7-4561-9887-1b956c8e6ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:00.056542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b549fc3-acd7-4561-9887-1b956c8e6ea1 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:00.057886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 720/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 22:22:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:22:00.063688 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afa624-9fdf-430d-ba9e-b17f77aab399 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:00.064138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afa624-9fdf-430d-ba9e-b17f77aab399 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:00.064243 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afa624-9fdf-430d-ba9e-b17f77aab399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:00.127294 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:00.130466 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-63afa624-9fdf-430d-ba9e-b17f77aab399 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:22:00.132122 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-63afa624-9fdf-430d-ba9e-b17f77aab399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:00.238891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 734/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 22:22:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:22:00.278191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b549fc3-acd7-4561-9887-1b956c8e6ea1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:22:00.280431 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6b549fc3-acd7-4561-9887-1b956c8e6ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:00.384735 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b549fc3-acd7-4561-9887-1b956c8e6ea1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:22:00.434303 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:00.459872 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977a3591-7c87-49d5-b1f3-1235af406ff0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:22:00.460491 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a3591-7c87-49d5-b1f3-1235af406ff0 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:00.460776 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a3591-7c87-49d5-b1f3-1235af406ff0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:00.461055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977a3591-7c87-49d5-b1f3-1235af406ff0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:00.462712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-977a3591-7c87-49d5-b1f3-1235af406ff0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:00.575186 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a3591-7c87-49d5-b1f3-1235af406ff0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:22:00.576776 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977a3591-7c87-49d5-b1f3-1235af406ff0 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:22:00.594908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 721/1455] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:22:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:22:02.637682 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc01a2c-2467-4105-86e1-d492ffb3cc84 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:02.638797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc01a2c-2467-4105-86e1-d492ffb3cc84 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:02.638797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc01a2c-2467-4105-86e1-d492ffb3cc84 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:02.639684 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 735/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 22:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:22:02.644866 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dca386-ae7a-4633-94c3-69c612f55005 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:02.645292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dca386-ae7a-4633-94c3-69c612f55005 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:02.645431 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dca386-ae7a-4633-94c3-69c612f55005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:02.711844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-35dca386-ae7a-4633-94c3-69c612f55005 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:22:02.713049 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-35dca386-ae7a-4633-94c3-69c612f55005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:02.744355 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:02.817876 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 722/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 22:22:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:22:02.856378 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc01a2c-2467-4105-86e1-d492ffb3cc84 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:22:02.858565 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-acc01a2c-2467-4105-86e1-d492ffb3cc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:02.960961 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc01a2c-2467-4105-86e1-d492ffb3cc84 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:22:02.989698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:03.034299 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5227cff-109e-49ad-bfc1-dcb5290ea80a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:22:03.034908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5227cff-109e-49ad-bfc1-dcb5290ea80a service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:03.035185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5227cff-109e-49ad-bfc1-dcb5290ea80a service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:03.035454 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5227cff-109e-49ad-bfc1-dcb5290ea80a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:03.037060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a5227cff-109e-49ad-bfc1-dcb5290ea80a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:03.152549 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5227cff-109e-49ad-bfc1-dcb5290ea80a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:22:03.155934 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5227cff-109e-49ad-bfc1-dcb5290ea80a service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:22:03.178216 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 736/1458] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:22:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:22:05.369777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf79fe-5dee-4c99-b3e1-5332c82b9543 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:05.370347 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf79fe-5dee-4c99-b3e1-5332c82b9543 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:05.370347 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf79fe-5dee-4c99-b3e1-5332c82b9543 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:05.371705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 723/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 22:22:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:22:05.377577 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8456e1-0a6d-4849-91c3-7c620bee0ce1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:05.378026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8456e1-0a6d-4849-91c3-7c620bee0ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:05.378139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8456e1-0a6d-4849-91c3-7c620bee0ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:05.444388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-db8456e1-0a6d-4849-91c3-7c620bee0ce1 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:22:05.445519 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-db8456e1-0a6d-4849-91c3-7c620bee0ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:05.454420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:05.543165 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 737/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 22:22:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:22:05.581251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf79fe-5dee-4c99-b3e1-5332c82b9543 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:22:05.582838 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-eadf79fe-5dee-4c99-b3e1-5332c82b9543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:05.683959 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf79fe-5dee-4c99-b3e1-5332c82b9543 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:22:05.738666 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:05.757006 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89ce4d-e1f4-419a-84ed-50b1cac6bd5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:22:05.757654 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89ce4d-e1f4-419a-84ed-50b1cac6bd5f service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:05.757938 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89ce4d-e1f4-419a-84ed-50b1cac6bd5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:05.758210 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89ce4d-e1f4-419a-84ed-50b1cac6bd5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:05.759807 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9e89ce4d-e1f4-419a-84ed-50b1cac6bd5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:05.867155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e89ce4d-e1f4-419a-84ed-50b1cac6bd5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:22:05.868588 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e89ce4d-e1f4-419a-84ed-50b1cac6bd5f service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:22:05.890130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 724/1461] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:22:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:22:08.083309 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cdd5d-064c-4591-8237-8cc2a27f7fb0 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:08.083872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cdd5d-064c-4591-8237-8cc2a27f7fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:08.083872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16cdd5d-064c-4591-8237-8cc2a27f7fb0 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:08.085217 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 738/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 22:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:22:08.090800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f98f32-afc7-4901-8e56-fc4fbc4454da None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:08.091221 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f98f32-afc7-4901-8e56-fc4fbc4454da None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:08.091418 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f98f32-afc7-4901-8e56-fc4fbc4454da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:08.173976 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d3f98f32-afc7-4901-8e56-fc4fbc4454da None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:22:08.175217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d3f98f32-afc7-4901-8e56-fc4fbc4454da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:08.222424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:08.302040 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 725/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 22:22:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:22:08.343069 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16cdd5d-064c-4591-8237-8cc2a27f7fb0 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:22:08.344599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f16cdd5d-064c-4591-8237-8cc2a27f7fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:08.461686 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16cdd5d-064c-4591-8237-8cc2a27f7fb0 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:22:08.499678 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:08.537172 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d93facc-7e47-456a-8242-3459a8d134be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:22:08.537843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93facc-7e47-456a-8242-3459a8d134be service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:08.538128 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93facc-7e47-456a-8242-3459a8d134be service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:08.538402 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d93facc-7e47-456a-8242-3459a8d134be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:08.540018 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5d93facc-7e47-456a-8242-3459a8d134be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:08.647061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93facc-7e47-456a-8242-3459a8d134be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:22:08.648597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d93facc-7e47-456a-8242-3459a8d134be service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:22:08.667581 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 739/1464] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:22:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:22:10.673623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98dc350-4e5e-41cf-8eb3-720b9ef91a8b None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:10.674194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98dc350-4e5e-41cf-8eb3-720b9ef91a8b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:10.674194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98dc350-4e5e-41cf-8eb3-720b9ef91a8b None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:10.675534 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 726/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 22:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:22:10.680873 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214a198-c8d6-4445-8f9c-5bf0ea099f23 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:10.681289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214a198-c8d6-4445-8f9c-5bf0ea099f23 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:10.681432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214a198-c8d6-4445-8f9c-5bf0ea099f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:10.747055 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3214a198-c8d6-4445-8f9c-5bf0ea099f23 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:22:10.748433 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3214a198-c8d6-4445-8f9c-5bf0ea099f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:10.766877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:10.844351 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 740/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 22:22:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:22:10.881604 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98dc350-4e5e-41cf-8eb3-720b9ef91a8b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:22:10.883179 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e98dc350-4e5e-41cf-8eb3-720b9ef91a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:11.106209 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98dc350-4e5e-41cf-8eb3-720b9ef91a8b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:22:11.172431 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:11.179960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f70079a-af2a-4084-a596-d204854e3148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:22:11.180632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70079a-af2a-4084-a596-d204854e3148 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:11.180915 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70079a-af2a-4084-a596-d204854e3148 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:11.181190 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70079a-af2a-4084-a596-d204854e3148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:11.182858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9f70079a-af2a-4084-a596-d204854e3148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:11.290310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70079a-af2a-4084-a596-d204854e3148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:22:11.291738 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f70079a-af2a-4084-a596-d204854e3148 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:22:11.309985 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 727/1467] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:22:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:22:13.321880 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c55361-9bdf-4732-b87c-133b0a899bd6 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:13.322475 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c55361-9bdf-4732-b87c-133b0a899bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:13.322475 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c55361-9bdf-4732-b87c-133b0a899bd6 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:13.323813 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 741/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 22:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:22:13.329364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66262e62-3b7c-4ed0-a22a-1761e25e162b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:13.329807 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66262e62-3b7c-4ed0-a22a-1761e25e162b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:13.329961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66262e62-3b7c-4ed0-a22a-1761e25e162b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:13.395591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-66262e62-3b7c-4ed0-a22a-1761e25e162b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:22:13.396620 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-66262e62-3b7c-4ed0-a22a-1761e25e162b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:13.444967 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:13.493125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 728/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 22:22:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:22:13.529588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c55361-9bdf-4732-b87c-133b0a899bd6 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:22:13.531108 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-39c55361-9bdf-4732-b87c-133b0a899bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:13.635695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c55361-9bdf-4732-b87c-133b0a899bd6 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:22:13.683728 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:13.710001 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ac7e46-b11c-4811-9941-40bc1d087026 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:22:13.710544 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac7e46-b11c-4811-9941-40bc1d087026 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:13.710825 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac7e46-b11c-4811-9941-40bc1d087026 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:13.711096 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ac7e46-b11c-4811-9941-40bc1d087026 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:13.712697 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c7ac7e46-b11c-4811-9941-40bc1d087026 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:13.818483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac7e46-b11c-4811-9941-40bc1d087026 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:22:13.819902 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ac7e46-b11c-4811-9941-40bc1d087026 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:22:13.843652 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 742/1470] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:22:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:22:23.491403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=22 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:23.840815 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=66 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:33.492891 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64b28b-d004-409f-8129-6fee89946ff8 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:33.493435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64b28b-d004-409f-8129-6fee89946ff8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:33.493435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64b28b-d004-409f-8129-6fee89946ff8 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:33.494893 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 729/1471] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:22:33.500063 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c501a08-2043-432f-b2ba-73a67503fd4d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:33.500530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c501a08-2043-432f-b2ba-73a67503fd4d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:33.500636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c501a08-2043-432f-b2ba-73a67503fd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:33.566630 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9c501a08-2043-432f-b2ba-73a67503fd4d None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:22:33.567853 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9c501a08-2043-432f-b2ba-73a67503fd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:33.664584 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 743/1472] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:22:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:22:33.746574 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ad973-f137-4296-a581-1f79230bf5c0 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:33.746824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ad973-f137-4296-a581-1f79230bf5c0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:33.747093 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946ad973-f137-4296-a581-1f79230bf5c0 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:33.748919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 730/1473] 10.222.0.39 () {54 vars in 966 bytes} [Thu Jul 28 22:22:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:22:33.753681 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9fed6a-4420-4a17-9a92-241eac4f9ceb None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:33.753987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9fed6a-4420-4a17-9a92-241eac4f9ceb None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:33.754248 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9fed6a-4420-4a17-9a92-241eac4f9ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:33.834368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-2e9fed6a-4420-4a17-9a92-241eac4f9ceb None None] MFA Rules not processed for user `0fecb440c0874ede9f3cf544cfed93ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:22:33.835590 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2e9fed6a-4420-4a17-9a92-241eac4f9ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:33.859673 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:33.915735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 744/1474] 10.222.0.39 () {58 vars in 1063 bytes} [Thu Jul 28 22:22:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:22:33.923576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ad973-f137-4296-a581-1f79230bf5c0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:22:33.925088 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-946ad973-f137-4296-a581-1f79230bf5c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:34.028602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946ad973-f137-4296-a581-1f79230bf5c0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:22:34.100575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df648ca-bca5-411f-9fb6-10dac509d784 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fecb440c0874ede9f3cf544cfed93ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:22:34.101191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df648ca-bca5-411f-9fb6-10dac509d784 service glance] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:34.101480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df648ca-bca5-411f-9fb6-10dac509d784 service glance] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:34.101785 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df648ca-bca5-411f-9fb6-10dac509d784 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:34.103396 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2df648ca-bca5-411f-9fb6-10dac509d784 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:34.159095 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:34.216781 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df648ca-bca5-411f-9fb6-10dac509d784 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:22:34.218189 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df648ca-bca5-411f-9fb6-10dac509d784 service glance] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:22:34.237057 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 731/1475] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:22:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:22:36.311923 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecab23-6c7e-44ed-817f-b4818d99a000 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:36.312491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecab23-6c7e-44ed-817f-b4818d99a000 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:36.312491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ecab23-6c7e-44ed-817f-b4818d99a000 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:36.313858 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 745/1476] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:22:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:22:36.319088 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fa690-44ab-49ab-a0fb-8cd374c9232b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:36.319538 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fa690-44ab-49ab-a0fb-8cd374c9232b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:36.319642 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419fa690-44ab-49ab-a0fb-8cd374c9232b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:36.391522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-419fa690-44ab-49ab-a0fb-8cd374c9232b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:22:36.392640 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-419fa690-44ab-49ab-a0fb-8cd374c9232b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:36.486585 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 732/1477] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:22:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:22:36.509850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecab23-6c7e-44ed-817f-b4818d99a000 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:22:36.511357 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-86ecab23-6c7e-44ed-817f-b4818d99a000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:36.635736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ecab23-6c7e-44ed-817f-b4818d99a000 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:22:36.684757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:36.728690 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce981cc1-8f01-4044-ab79-726efa2dce66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:22:36.729312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce981cc1-8f01-4044-ab79-726efa2dce66 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:36.729659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce981cc1-8f01-4044-ab79-726efa2dce66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:36.729936 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce981cc1-8f01-4044-ab79-726efa2dce66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:36.730632 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce981cc1-8f01-4044-ab79-726efa2dce66 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:22:36.732957 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-ce981cc1-8f01-4044-ab79-726efa2dce66 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 22:22:36.732957 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Jul 28 22:22:36.732957 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: Jul 28 22:22:36.734139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce981cc1-8f01-4044-ab79-726efa2dce66 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:22:36.753168 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 746/1478] 10.222.0.39 () {56 vars in 1108 bytes} [Thu Jul 28 22:22:36 2022] GET /identity/v3/projects => generated 2553 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:22:38.766691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b8719-3d7e-4cc4-ac20-6b2a1098ae23 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:38.767253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b8719-3d7e-4cc4-ac20-6b2a1098ae23 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:38.767434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b8719-3d7e-4cc4-ac20-6b2a1098ae23 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:38.768973 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 733/1479] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:22:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:22:38.774444 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ae808-7386-44b1-ab0b-2ffa5d3e8b75 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:38.774877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ae808-7386-44b1-ab0b-2ffa5d3e8b75 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:38.775033 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ae808-7386-44b1-ab0b-2ffa5d3e8b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:38.840399 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5a4ae808-7386-44b1-ab0b-2ffa5d3e8b75 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:22:38.841631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5a4ae808-7386-44b1-ab0b-2ffa5d3e8b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:38.937990 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 747/1480] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:22:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:22:38.980808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b8719-3d7e-4cc4-ac20-6b2a1098ae23 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:22:38.982550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-450b8719-3d7e-4cc4-ac20-6b2a1098ae23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:38.992421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:39.128892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b8719-3d7e-4cc4-ac20-6b2a1098ae23 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:22:39.208676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:39.241583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5bdd20-8f4a-4550-83af-f33e72472927 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:22:39.242197 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bdd20-8f4a-4550-83af-f33e72472927 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:39.242477 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bdd20-8f4a-4550-83af-f33e72472927 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:39.242748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5bdd20-8f4a-4550-83af-f33e72472927 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:39.244360 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5e5bdd20-8f4a-4550-83af-f33e72472927 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:39.331628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5bdd20-8f4a-4550-83af-f33e72472927 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:22:39.333210 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5bdd20-8f4a-4550-83af-f33e72472927 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:22:39.350900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 734/1481] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:22:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:22:41.543259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd791c80-62fc-42a2-bd7e-42d7cfe7ab0f None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:41.543787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd791c80-62fc-42a2-bd7e-42d7cfe7ab0f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:41.543787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd791c80-62fc-42a2-bd7e-42d7cfe7ab0f None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:41.545206 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 748/1482] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:22:41.550364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a0625-e941-47fe-99a8-83e47bfd54c6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:41.550791 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a0625-e941-47fe-99a8-83e47bfd54c6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:41.550908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a0625-e941-47fe-99a8-83e47bfd54c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:41.616142 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ea6a0625-e941-47fe-99a8-83e47bfd54c6 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:22:41.617354 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ea6a0625-e941-47fe-99a8-83e47bfd54c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:41.663402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:41.712283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 735/1483] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:22:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:22:41.748875 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd791c80-62fc-42a2-bd7e-42d7cfe7ab0f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:22:41.750441 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fd791c80-62fc-42a2-bd7e-42d7cfe7ab0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:41.774354 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:41.854021 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd791c80-62fc-42a2-bd7e-42d7cfe7ab0f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:22:41.926826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5053496d-4a51-475f-87a7-a122c25ae8c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:22:41.927442 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053496d-4a51-475f-87a7-a122c25ae8c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:41.927723 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053496d-4a51-475f-87a7-a122c25ae8c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:41.927995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5053496d-4a51-475f-87a7-a122c25ae8c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:41.929692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5053496d-4a51-475f-87a7-a122c25ae8c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:42.003396 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:42.038041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053496d-4a51-475f-87a7-a122c25ae8c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:22:42.039474 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5053496d-4a51-475f-87a7-a122c25ae8c7 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:22:42.058066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 749/1484] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:22:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:22:44.117352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e286f-4e13-48bc-bd0a-4523793acd98 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:44.118046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e286f-4e13-48bc-bd0a-4523793acd98 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:44.118046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e286f-4e13-48bc-bd0a-4523793acd98 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:44.119266 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 736/1485] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:22:44.124974 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ea2e5-fd22-4eed-bb5e-6fb1f48fe404 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:44.125386 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ea2e5-fd22-4eed-bb5e-6fb1f48fe404 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:44.125589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564ea2e5-fd22-4eed-bb5e-6fb1f48fe404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:44.192994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-564ea2e5-fd22-4eed-bb5e-6fb1f48fe404 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:22:44.194247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-564ea2e5-fd22-4eed-bb5e-6fb1f48fe404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:44.292004 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 750/1486] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:22:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:22:44.331558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e286f-4e13-48bc-bd0a-4523793acd98 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:22:44.333136 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-858e286f-4e13-48bc-bd0a-4523793acd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:44.440003 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e286f-4e13-48bc-bd0a-4523793acd98 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:22:44.518998 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:44.521185 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c23348-3941-4dd9-a767-174b2a801964 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:22:44.522050 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c23348-3941-4dd9-a767-174b2a801964 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:44.522500 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c23348-3941-4dd9-a767-174b2a801964 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:44.522969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c23348-3941-4dd9-a767-174b2a801964 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:44.524611 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-81c23348-3941-4dd9-a767-174b2a801964 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:44.634050 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c23348-3941-4dd9-a767-174b2a801964 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:22:44.635501 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c23348-3941-4dd9-a767-174b2a801964 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:22:44.653624 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 737/1487] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:22:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:22:46.852660 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457845c-325d-4ccd-aafc-d512ac6135b6 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:46.853123 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457845c-325d-4ccd-aafc-d512ac6135b6 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:46.853235 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457845c-325d-4ccd-aafc-d512ac6135b6 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:46.854634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 751/1488] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:22:46.860444 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfbe17-f896-4db7-9cc7-986b1bcf4038 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:46.860882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfbe17-f896-4db7-9cc7-986b1bcf4038 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:46.860994 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfbe17-f896-4db7-9cc7-986b1bcf4038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:46.926397 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-40dfbe17-f896-4db7-9cc7-986b1bcf4038 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:22:46.927623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-40dfbe17-f896-4db7-9cc7-986b1bcf4038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:46.978426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:47.022351 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 738/1489] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:22:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:22:47.063387 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457845c-325d-4ccd-aafc-d512ac6135b6 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:22:47.064900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4457845c-325d-4ccd-aafc-d512ac6135b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:47.100871 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:47.169224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4457845c-325d-4ccd-aafc-d512ac6135b6 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:22:47.242791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168719d9-6c5c-40d3-bd73-20de81f5e09c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:22:47.243411 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168719d9-6c5c-40d3-bd73-20de81f5e09c service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:47.243695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168719d9-6c5c-40d3-bd73-20de81f5e09c service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:47.243997 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168719d9-6c5c-40d3-bd73-20de81f5e09c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:47.245650 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-168719d9-6c5c-40d3-bd73-20de81f5e09c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:47.324875 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:47.354139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168719d9-6c5c-40d3-bd73-20de81f5e09c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:22:47.355520 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168719d9-6c5c-40d3-bd73-20de81f5e09c service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:22:47.374536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 752/1490] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:22:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:22:49.616174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4efde2-71b0-445d-a0d7-a2f27ad2d521 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:49.616698 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4efde2-71b0-445d-a0d7-a2f27ad2d521 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:49.616819 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4efde2-71b0-445d-a0d7-a2f27ad2d521 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:49.618115 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 739/1491] 10.222.0.39 () {54 vars in 926 bytes} [Thu Jul 28 22:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:22:49.623485 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552fb54-d283-45e1-920e-f117f6d344a1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:49.623929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552fb54-d283-45e1-920e-f117f6d344a1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:49.624025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552fb54-d283-45e1-920e-f117f6d344a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:49.688961 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a552fb54-d283-45e1-920e-f117f6d344a1 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:22:49.690182 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a552fb54-d283-45e1-920e-f117f6d344a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:49.786616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 753/1492] 10.222.0.39 () {58 vars in 1023 bytes} [Thu Jul 28 22:22:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:22:49.826168 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4efde2-71b0-445d-a0d7-a2f27ad2d521 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:22:49.827690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3d4efde2-71b0-445d-a0d7-a2f27ad2d521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:49.929060 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4efde2-71b0-445d-a0d7-a2f27ad2d521 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:22:49.999925 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654dd2e5-f3af-4925-9d00-c7e67a5197f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:22:50.000535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654dd2e5-f3af-4925-9d00-c7e67a5197f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:22:50.000814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654dd2e5-f3af-4925-9d00-c7e67a5197f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:22:50.001085 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654dd2e5-f3af-4925-9d00-c7e67a5197f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:22:50.002705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-654dd2e5-f3af-4925-9d00-c7e67a5197f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:22:50.011537 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:22:50.111116 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654dd2e5-f3af-4925-9d00-c7e67a5197f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:22:50.112672 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654dd2e5-f3af-4925-9d00-c7e67a5197f3 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:22:50.135255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 740/1493] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:22:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:22:59.784763 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=82 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:23:00.132920 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=50 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:24:18.771425 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d0fe8-58ce-4102-9c52-17cde7fad093 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:24:18.771891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d0fe8-58ce-4102-9c52-17cde7fad093 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:24:18.771997 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d0fe8-58ce-4102-9c52-17cde7fad093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:24:18.848801 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-729d0fe8-58ce-4102-9c52-17cde7fad093 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:24:18.850550 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-729d0fe8-58ce-4102-9c52-17cde7fad093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:24:18.948147 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 754/1494] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:24:18.956073 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179c6bd-dd5a-424d-b401-ba8c4aad848b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:24:18.956558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179c6bd-dd5a-424d-b401-ba8c4aad848b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:24:18.956663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179c6bd-dd5a-424d-b401-ba8c4aad848b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:24:19.033384 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2179c6bd-dd5a-424d-b401-ba8c4aad848b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:24:19.034561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2179c6bd-dd5a-424d-b401-ba8c4aad848b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:24:19.129984 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 741/1495] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:24:19.135095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d0fe8-58ce-4102-9c52-17cde7fad093 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:24:19.136765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-729d0fe8-58ce-4102-9c52-17cde7fad093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:24:19.148125 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d0fe8-58ce-4102-9c52-17cde7fad093 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:24:19.208494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:24:19.242459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0a7ca-7718-41f8-b3c1-6108fb4ec37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:24:19.243075 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0a7ca-7718-41f8-b3c1-6108fb4ec37c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:24:19.243356 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0a7ca-7718-41f8-b3c1-6108fb4ec37c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:24:19.243631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb0a7ca-7718-41f8-b3c1-6108fb4ec37c admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:24:19.244477 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb0a7ca-7718-41f8-b3c1-6108fb4ec37c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:24:19.244856 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb0a7ca-7718-41f8-b3c1-6108fb4ec37c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:24:19.245897 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-cdb0a7ca-7718-41f8-b3c1-6108fb4ec37c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 22:24:19.245897 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Jul 28 22:24:19.245897 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: Jul 28 22:24:19.246789 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb0a7ca-7718-41f8-b3c1-6108fb4ec37c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:24:19.257936 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 755/1496] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:24:19.266905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179c6bd-dd5a-424d-b401-ba8c4aad848b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:24:19.276854 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179c6bd-dd5a-424d-b401-ba8c4aad848b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:24:19.337043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:24:19.370739 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e6296-33bf-4466-9cbf-4d42eb16cc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:24:19.371350 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e6296-33bf-4466-9cbf-4d42eb16cc39 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:24:19.371647 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e6296-33bf-4466-9cbf-4d42eb16cc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:24:19.371900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e6296-33bf-4466-9cbf-4d42eb16cc39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:24:19.372560 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e6296-33bf-4466-9cbf-4d42eb16cc39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:24:19.374311 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-808e6296-33bf-4466-9cbf-4d42eb16cc39 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 22:24:19.374311 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jul 28 22:24:19.374311 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: Jul 28 22:24:19.375508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e6296-33bf-4466-9cbf-4d42eb16cc39 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:24:19.429616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 742/1497] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:24:19 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:24:19.434288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0a7ca-7718-41f8-b3c1-6108fb4ec37c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:24:19.435794 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-cdb0a7ca-7718-41f8-b3c1-6108fb4ec37c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:24:19.468961 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb0a7ca-7718-41f8-b3c1-6108fb4ec37c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:24:19.560946 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843ae78-11aa-492e-93e9-fd6e8d8cfcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:24:19.561583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843ae78-11aa-492e-93e9-fd6e8d8cfcc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:24:19.561868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843ae78-11aa-492e-93e9-fd6e8d8cfcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:24:19.562140 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843ae78-11aa-492e-93e9-fd6e8d8cfcc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:24:19.562804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843ae78-11aa-492e-93e9-fd6e8d8cfcc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:24:19.563802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-a843ae78-11aa-492e-93e9-fd6e8d8cfcc5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 22:24:19.563802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Jul 28 22:24:19.563802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: Jul 28 22:24:19.565006 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843ae78-11aa-492e-93e9-fd6e8d8cfcc5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:24:19.607026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 756/1498] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:24:19 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:24:19.611142 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e6296-33bf-4466-9cbf-4d42eb16cc39 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:24:19.612625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-808e6296-33bf-4466-9cbf-4d42eb16cc39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:24:19.675138 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e6296-33bf-4466-9cbf-4d42eb16cc39 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:24:19.760010 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:24:19.772289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f5d38-3074-4e88-869c-d88da70c30bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:24:19.772909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f5d38-3074-4e88-869c-d88da70c30bc admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:24:19.773206 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f5d38-3074-4e88-869c-d88da70c30bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:24:19.773474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f5d38-3074-4e88-869c-d88da70c30bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:24:19.774192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1f5d38-3074-4e88-869c-d88da70c30bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:24:19.775912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-0b1f5d38-3074-4e88-869c-d88da70c30bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 22:24:19.775912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jul 28 22:24:19.775912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: Jul 28 22:24:19.777576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1f5d38-3074-4e88-869c-d88da70c30bc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:24:19.789859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 743/1499] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:24:19.793884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843ae78-11aa-492e-93e9-fd6e8d8cfcc5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:24:19.795387 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a843ae78-11aa-492e-93e9-fd6e8d8cfcc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:24:19.807232 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843ae78-11aa-492e-93e9-fd6e8d8cfcc5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:24:19.832778 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:24:19.904357 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752713b7-992d-4067-8414-6dd39396ecaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:24:19.905279 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752713b7-992d-4067-8414-6dd39396ecaf admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:24:19.905703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752713b7-992d-4067-8414-6dd39396ecaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:24:19.905984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752713b7-992d-4067-8414-6dd39396ecaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:24:19.906684 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752713b7-992d-4067-8414-6dd39396ecaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:24:19.907702 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-752713b7-992d-4067-8414-6dd39396ecaf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 22:24:19.907702 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Jul 28 22:24:19.907702 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: Jul 28 22:24:19.908599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752713b7-992d-4067-8414-6dd39396ecaf admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:24:19.921280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 757/1500] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:24:19.925470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f5d38-3074-4e88-869c-d88da70c30bc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:24:19.926995 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0b1f5d38-3074-4e88-869c-d88da70c30bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:24:19.959746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1f5d38-3074-4e88-869c-d88da70c30bc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:24:20.052097 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1592ba-384c-445d-81dc-7215803b96b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:24:20.052782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1592ba-384c-445d-81dc-7215803b96b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:24:20.053057 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1592ba-384c-445d-81dc-7215803b96b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:24:20.053323 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1592ba-384c-445d-81dc-7215803b96b9 admin admin] PATH_INFO: `/v3/projects/4fa2f7f894cc435b878f1b06043e5274/users/ed4a327710234bb59363d2ebce3fb4c1/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:24:20.075615 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1592ba-384c-445d-81dc-7215803b96b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fa2f7f894cc435b878f1b06043e5274, user_id=ed4a327710234bb59363d2ebce3fb4c1, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:24:20.076647 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-fc1592ba-384c-445d-81dc-7215803b96b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 22:24:20.076647 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jul 28 22:24:20.076647 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: Jul 28 22:24:20.078641 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1592ba-384c-445d-81dc-7215803b96b9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:24:20.102805 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 744/1501] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:24:19 2022] PUT /identity/v3/projects/4fa2f7f894cc435b878f1b06043e5274/users/ed4a327710234bb59363d2ebce3fb4c1/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:24:20.115360 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310fd29-247c-475e-a1ef-478cd91663e0 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:24:20.115635 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310fd29-247c-475e-a1ef-478cd91663e0 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:24:20.115911 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310fd29-247c-475e-a1ef-478cd91663e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:24:20.176574 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-2310fd29-247c-475e-a1ef-478cd91663e0 None None] MFA Rules not processed for user `ed4a327710234bb59363d2ebce3fb4c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:24:20.177824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2310fd29-247c-475e-a1ef-478cd91663e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:24:20.264189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 758/1502] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:24:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:24:20.278053 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f733d0-fcaf-4e65-921c-f182843669ca None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:24:20.278338 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f733d0-fcaf-4e65-921c-f182843669ca None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:24:20.278610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f733d0-fcaf-4e65-921c-f182843669ca None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:24:20.279967 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 745/1503] 10.222.0.39 () {54 vars in 964 bytes} [Thu Jul 28 22:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:24:20.285019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16dffc-6744-4ab1-ad08-871ced44dbc1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:24:20.285330 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16dffc-6744-4ab1-ad08-871ced44dbc1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:24:20.285638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a16dffc-6744-4ab1-ad08-871ced44dbc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:24:20.316351 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:24:20.362304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-4a16dffc-6744-4ab1-ad08-871ced44dbc1 None None] MFA Rules not processed for user `b4aa320b18ce43c3acfece4546fea0f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:24:20.363519 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4a16dffc-6744-4ab1-ad08-871ced44dbc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:24:20.442270 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 759/1504] 10.222.0.39 () {58 vars in 1061 bytes} [Thu Jul 28 22:24:20 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:24:20.450602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f733d0-fcaf-4e65-921c-f182843669ca None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:24:20.452167 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a4f733d0-fcaf-4e65-921c-f182843669ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:24:20.502805 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:24:20.558755 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f733d0-fcaf-4e65-921c-f182843669ca None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:24:20.631375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844f20b-d744-4973-a639-dfe5e48abba0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:24:20.631989 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844f20b-d744-4973-a639-dfe5e48abba0 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:24:20.632274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844f20b-d744-4973-a639-dfe5e48abba0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:24:20.632545 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844f20b-d744-4973-a639-dfe5e48abba0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:24:20.634185 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f844f20b-d744-4973-a639-dfe5e48abba0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:24:20.732741 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844f20b-d744-4973-a639-dfe5e48abba0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:24:20.734151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844f20b-d744-4973-a639-dfe5e48abba0 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:24:20.737662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:24:20.753350 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 746/1505] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:24:20 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:24:20.798822 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f9b90-13a3-465e-a630-20862ff8caf8 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:24:20.799218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f9b90-13a3-465e-a630-20862ff8caf8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:24:20.799374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f9b90-13a3-465e-a630-20862ff8caf8 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:24:20.800645 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 760/1506] 10.222.0.39 () {52 vars in 830 bytes} [Thu Jul 28 22:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:24:20.804178 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844f20b-d744-4973-a639-dfe5e48abba0 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:24:20.805403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f844f20b-d744-4973-a639-dfe5e48abba0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:24:20.913131 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844f20b-d744-4973-a639-dfe5e48abba0 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:24:21.005450 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38440c8-47d1-4ada-9b81-763107e13204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:24:21.006130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38440c8-47d1-4ada-9b81-763107e13204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:24:21.006304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38440c8-47d1-4ada-9b81-763107e13204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:24:21.006572 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38440c8-47d1-4ada-9b81-763107e13204 admin admin] PATH_INFO: `/v3/users/ed4a327710234bb59363d2ebce3fb4c1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:24:21.018938 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38440c8-47d1-4ada-9b81-763107e13204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed4a327710234bb59363d2ebce3fb4c1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:24:21.019987 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-a38440c8-47d1-4ada-9b81-763107e13204 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 22:24:21.019987 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jul 28 22:24:21.019987 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: Jul 28 22:24:21.021199 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38440c8-47d1-4ada-9b81-763107e13204 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:24:21.031105 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:24:21.084854 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a38440c8-47d1-4ada-9b81-763107e13204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed4a327710234bb59363d2ebce3fb4c1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:24:21.097214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a38440c8-47d1-4ada-9b81-763107e13204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed4a327710234bb59363d2ebce3fb4c1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:24:21.097675 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a38440c8-47d1-4ada-9b81-763107e13204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed4a327710234bb59363d2ebce3fb4c1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:24:21.098188 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a38440c8-47d1-4ada-9b81-763107e13204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed4a327710234bb59363d2ebce3fb4c1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:24:21.116654 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a38440c8-47d1-4ada-9b81-763107e13204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed4a327710234bb59363d2ebce3fb4c1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:24:21.130714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 747/1507] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:24:20 2022] DELETE /identity/v3/users/ed4a327710234bb59363d2ebce3fb4c1 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:24:21.135245 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cb974-6f8b-4686-b65c-31b65d67d1f8 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:24:21.135539 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cb974-6f8b-4686-b65c-31b65d67d1f8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:24:21.135821 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cb974-6f8b-4686-b65c-31b65d67d1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:24:21.194140 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-686cb974-6f8b-4686-b65c-31b65d67d1f8 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:24:21.195350 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-686cb974-6f8b-4686-b65c-31b65d67d1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:24:21.270547 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:24:21.284192 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 761/1508] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:24:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:24:21.294487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38440c8-47d1-4ada-9b81-763107e13204 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:24:21.295731 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a38440c8-47d1-4ada-9b81-763107e13204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:24:21.395085 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38440c8-47d1-4ada-9b81-763107e13204 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:24:21.451658 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:24:21.472720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da344d3-afdf-4e42-90ae-6c544d3a817f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:24:21.473336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da344d3-afdf-4e42-90ae-6c544d3a817f service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:24:21.473650 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da344d3-afdf-4e42-90ae-6c544d3a817f service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:24:21.473927 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da344d3-afdf-4e42-90ae-6c544d3a817f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:24:21.475543 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6da344d3-afdf-4e42-90ae-6c544d3a817f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:24:21.585603 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da344d3-afdf-4e42-90ae-6c544d3a817f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:24:21.586958 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da344d3-afdf-4e42-90ae-6c544d3a817f service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:24:21.606344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 748/1509] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:24:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:24:21.958594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cb974-6f8b-4686-b65c-31b65d67d1f8 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:24:21.959854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-686cb974-6f8b-4686-b65c-31b65d67d1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:24:21.972111 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cb974-6f8b-4686-b65c-31b65d67d1f8 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:24:22.071375 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503a0be-b52e-42c7-980f-390138e9bd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:24:22.072036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503a0be-b52e-42c7-980f-390138e9bd33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:24:22.072310 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503a0be-b52e-42c7-980f-390138e9bd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:24:22.072776 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503a0be-b52e-42c7-980f-390138e9bd33 admin admin] PATH_INFO: `/v3/projects/4fa2f7f894cc435b878f1b06043e5274` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:24:22.082932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4503a0be-b52e-42c7-980f-390138e9bd33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fa2f7f894cc435b878f1b06043e5274)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:24:22.083956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-4503a0be-b52e-42c7-980f-390138e9bd33 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 22:24:22.083956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Jul 28 22:24:22.083956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: Jul 28 22:24:22.085165 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4503a0be-b52e-42c7-980f-390138e9bd33 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:24:22.143677 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4503a0be-b52e-42c7-980f-390138e9bd33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fa2f7f894cc435b878f1b06043e5274', 'request_id': 'req-4503a0be-b52e-42c7-980f-390138e9bd33'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:24:22.153706 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4503a0be-b52e-42c7-980f-390138e9bd33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fa2f7f894cc435b878f1b06043e5274', 'request_id': 'req-4503a0be-b52e-42c7-980f-390138e9bd33'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:24:22.160090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4503a0be-b52e-42c7-980f-390138e9bd33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fa2f7f894cc435b878f1b06043e5274', 'request_id': 'req-4503a0be-b52e-42c7-980f-390138e9bd33'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:24:22.190685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4503a0be-b52e-42c7-980f-390138e9bd33 admin admin] The token cache is being invalidate because project 4fa2f7f894cc435b878f1b06043e5274 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:24:22.190884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4503a0be-b52e-42c7-980f-390138e9bd33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:24:22.191425 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4503a0be-b52e-42c7-980f-390138e9bd33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:24:22.192787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 762/1510] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:24:21 2022] DELETE /identity/v3/projects/4fa2f7f894cc435b878f1b06043e5274 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:24:31.604060 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=62 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:24:32.188486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:26:00.188781 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fc869-11ad-474e-a0dd-39ecca29cae1 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:00.189375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fc869-11ad-474e-a0dd-39ecca29cae1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:00.189375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0fc869-11ad-474e-a0dd-39ecca29cae1 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:00.190980 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 749/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 22:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:26:00.195740 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb0eb0-95fd-4765-bade-3acacc5fd25b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:00.196154 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb0eb0-95fd-4765-bade-3acacc5fd25b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:00.196620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb0eb0-95fd-4765-bade-3acacc5fd25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:00.287429 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e7cb0eb0-95fd-4765-bade-3acacc5fd25b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:26:00.288642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e7cb0eb0-95fd-4765-bade-3acacc5fd25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:26:00.397930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 763/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 22:26:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:26:00.550212 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fc869-11ad-474e-a0dd-39ecca29cae1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:26:00.552317 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-be0fc869-11ad-474e-a0dd-39ecca29cae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:26:00.671947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0fc869-11ad-474e-a0dd-39ecca29cae1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:26:00.677275 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a6199a-9a7c-4c29-9b83-fd8f937761ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:26:00.677925 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6199a-9a7c-4c29-9b83-fd8f937761ef service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:00.678209 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6199a-9a7c-4c29-9b83-fd8f937761ef service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:00.678485 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a6199a-9a7c-4c29-9b83-fd8f937761ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:00.687550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6199a-9a7c-4c29-9b83-fd8f937761ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:26:00.689120 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a6199a-9a7c-4c29-9b83-fd8f937761ef service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:26:00.699439 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 750/1513] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:26:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:26:01.548919 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718a9da-22ad-4969-bc0d-8d35525d1bae None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:01.549456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718a9da-22ad-4969-bc0d-8d35525d1bae None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:01.549456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6718a9da-22ad-4969-bc0d-8d35525d1bae None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:01.550775 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 764/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 22:26:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:26:01.556076 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f3a17-9174-4fb0-839c-f88fa09a9421 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:01.556501 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f3a17-9174-4fb0-839c-f88fa09a9421 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:01.556698 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f3a17-9174-4fb0-839c-f88fa09a9421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:01.577471 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2a4f3a17-9174-4fb0-839c-f88fa09a9421 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:26:01.578695 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2a4f3a17-9174-4fb0-839c-f88fa09a9421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:26:01.586558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 751/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 22:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:26:01.743005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718a9da-22ad-4969-bc0d-8d35525d1bae None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:26:01.756982 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6718a9da-22ad-4969-bc0d-8d35525d1bae None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:26:01.762876 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1796091-0c68-43e0-b5b0-7d41ec6d8729 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:26:01.763484 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1796091-0c68-43e0-b5b0-7d41ec6d8729 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:01.763763 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1796091-0c68-43e0-b5b0-7d41ec6d8729 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:01.764035 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1796091-0c68-43e0-b5b0-7d41ec6d8729 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:01.771988 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1796091-0c68-43e0-b5b0-7d41ec6d8729 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:26:01.773824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1796091-0c68-43e0-b5b0-7d41ec6d8729 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:26:01.782984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 765/1516] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:26:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:26:02.392126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e389ab-dd85-45ad-bca2-fb454bbc2d57 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:02.392415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e389ab-dd85-45ad-bca2-fb454bbc2d57 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:02.392692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e389ab-dd85-45ad-bca2-fb454bbc2d57 None None] PATH_INFO: `/v3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:02.394250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 752/1517] 10.222.0.58 () {54 vars in 925 bytes} [Thu Jul 28 22:26:02 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:26:02.407944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6102d1-798f-4332-a1ad-eba02588fa1b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:02.408171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6102d1-798f-4332-a1ad-eba02588fa1b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:02.408447 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6102d1-798f-4332-a1ad-eba02588fa1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:02.468052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-db6102d1-798f-4332-a1ad-eba02588fa1b None None] MFA Rules not processed for user `fe003cd7bfbe4b92b1d64d300c3e7d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:26:02.469288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-db6102d1-798f-4332-a1ad-eba02588fa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:26:02.489269 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:26:02.545194 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 766/1518] 10.222.0.58 () {58 vars in 1015 bytes} [Thu Jul 28 22:26:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:26:06.480540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2965e8-c8df-4ddf-a3ab-21ee44475f77 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:06.481120 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2965e8-c8df-4ddf-a3ab-21ee44475f77 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:06.481120 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2965e8-c8df-4ddf-a3ab-21ee44475f77 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:06.482621 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 753/1519] 10.222.0.39 () {54 vars in 961 bytes} [Thu Jul 28 22:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:26:06.487498 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c3aa1-2052-4b6f-a7de-a7120fb66193 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:06.487948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c3aa1-2052-4b6f-a7de-a7120fb66193 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:06.488085 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379c3aa1-2052-4b6f-a7de-a7120fb66193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:06.508949 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-379c3aa1-2052-4b6f-a7de-a7120fb66193 None None] MFA Rules not processed for user `fe003cd7bfbe4b92b1d64d300c3e7d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:26:06.510186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-379c3aa1-2052-4b6f-a7de-a7120fb66193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:26:06.518884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 767/1520] 10.222.0.39 () {58 vars in 1058 bytes} [Thu Jul 28 22:26:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:26:06.526737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2965e8-c8df-4ddf-a3ab-21ee44475f77 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:26:06.540851 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2965e8-c8df-4ddf-a3ab-21ee44475f77 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:26:06.546592 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164cf066-d060-4578-b933-ec187d3c7782 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe003cd7bfbe4b92b1d64d300c3e7d8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:26:06.547214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cf066-d060-4578-b933-ec187d3c7782 service placement] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:06.547592 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cf066-d060-4578-b933-ec187d3c7782 service placement] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:06.547863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164cf066-d060-4578-b933-ec187d3c7782 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:06.557371 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cf066-d060-4578-b933-ec187d3c7782 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:26:06.559069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164cf066-d060-4578-b933-ec187d3c7782 service placement] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:26:06.571536 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 754/1521] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:26:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:26:13.022280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67062e73-3d07-4dbe-8eb4-525e549f25fa None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:13.022866 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67062e73-3d07-4dbe-8eb4-525e549f25fa None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:13.022866 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67062e73-3d07-4dbe-8eb4-525e549f25fa None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:13.025162 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 768/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 22:26:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:26:13.030770 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9a464-3e4f-4457-9bbe-5f79c6590c83 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:13.031241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9a464-3e4f-4457-9bbe-5f79c6590c83 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:13.031375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9a464-3e4f-4457-9bbe-5f79c6590c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:13.050687 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d5c9a464-3e4f-4457-9bbe-5f79c6590c83 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:26:13.051951 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d5c9a464-3e4f-4457-9bbe-5f79c6590c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:26:13.060591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 755/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 22:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:26:13.245547 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67062e73-3d07-4dbe-8eb4-525e549f25fa None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:26:13.262003 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67062e73-3d07-4dbe-8eb4-525e549f25fa None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:26:13.267922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1af3ab-dfbf-4900-875d-de13f6cd2ae5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:26:13.268537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1af3ab-dfbf-4900-875d-de13f6cd2ae5 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:13.268821 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1af3ab-dfbf-4900-875d-de13f6cd2ae5 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:13.269098 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1af3ab-dfbf-4900-875d-de13f6cd2ae5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:13.279906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1af3ab-dfbf-4900-875d-de13f6cd2ae5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:26:13.281450 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1af3ab-dfbf-4900-875d-de13f6cd2ae5 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:26:13.293098 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 769/1524] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:26:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:26:14.127785 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d39980-a157-40ce-9f0c-3d6c25acc075 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:14.128362 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d39980-a157-40ce-9f0c-3d6c25acc075 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:14.128362 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d39980-a157-40ce-9f0c-3d6c25acc075 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:14.129712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 756/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 28 22:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:26:14.134730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51645e-8afa-48f6-a7eb-dc45c5a385de None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:14.135178 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51645e-8afa-48f6-a7eb-dc45c5a385de None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:14.135321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51645e-8afa-48f6-a7eb-dc45c5a385de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:14.156543 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9d51645e-8afa-48f6-a7eb-dc45c5a385de None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:26:14.157795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9d51645e-8afa-48f6-a7eb-dc45c5a385de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:26:14.166911 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 770/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 28 22:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:26:14.320870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d39980-a157-40ce-9f0c-3d6c25acc075 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:26:14.335373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d39980-a157-40ce-9f0c-3d6c25acc075 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:26:14.341872 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7af336-2be9-420e-a991-b0736aceee29 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:26:14.342475 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7af336-2be9-420e-a991-b0736aceee29 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:14.342760 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7af336-2be9-420e-a991-b0736aceee29 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:14.343037 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7af336-2be9-420e-a991-b0736aceee29 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:14.351116 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7af336-2be9-420e-a991-b0736aceee29 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:26:14.353791 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7af336-2be9-420e-a991-b0736aceee29 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:26:14.363092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 757/1527] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:26:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:26:18.222556 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dbfbd-366c-46fe-ba25-26ba9e04c2f5 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:18.223066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dbfbd-366c-46fe-ba25-26ba9e04c2f5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:18.223185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337dbfbd-366c-46fe-ba25-26ba9e04c2f5 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:18.224524 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 771/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 22:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:26:18.229834 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db8b26-c44f-434a-859f-f79803fa983c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:18.230258 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db8b26-c44f-434a-859f-f79803fa983c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:18.230397 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41db8b26-c44f-434a-859f-f79803fa983c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:18.249325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-41db8b26-c44f-434a-859f-f79803fa983c None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:26:18.250544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-41db8b26-c44f-434a-859f-f79803fa983c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:26:18.258909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 758/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 22:26:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:26:18.331259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dbfbd-366c-46fe-ba25-26ba9e04c2f5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:26:18.333406 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-337dbfbd-366c-46fe-ba25-26ba9e04c2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:26:18.416325 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337dbfbd-366c-46fe-ba25-26ba9e04c2f5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:26:18.419783 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634e031-149b-41e1-ab1b-67c9ab3a9c70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:26:18.420399 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634e031-149b-41e1-ab1b-67c9ab3a9c70 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:18.420682 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634e031-149b-41e1-ab1b-67c9ab3a9c70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:18.420958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634e031-149b-41e1-ab1b-67c9ab3a9c70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:18.431048 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0634e031-149b-41e1-ab1b-67c9ab3a9c70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:26:18.432486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0634e031-149b-41e1-ab1b-67c9ab3a9c70 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:26:18.441819 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 772/1530] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:26:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:26:20.446026 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30018a-680c-44c0-a2e9-2a94ce675d05 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:20.446533 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30018a-680c-44c0-a2e9-2a94ce675d05 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:20.446533 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30018a-680c-44c0-a2e9-2a94ce675d05 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:20.447848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 759/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 22:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:26:20.453151 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7b3c9-8675-4eaa-865c-cdb8c6dc9c5f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:20.453624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7b3c9-8675-4eaa-865c-cdb8c6dc9c5f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:20.453724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f7b3c9-8675-4eaa-865c-cdb8c6dc9c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:20.473625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-34f7b3c9-8675-4eaa-865c-cdb8c6dc9c5f None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:26:20.475186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-34f7b3c9-8675-4eaa-865c-cdb8c6dc9c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:26:20.483946 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 773/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 22:26:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:26:20.522399 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30018a-680c-44c0-a2e9-2a94ce675d05 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:26:20.533348 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30018a-680c-44c0-a2e9-2a94ce675d05 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:26:20.536813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf3fb2b-4493-4297-951d-f97e62a33f95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:26:20.537415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3fb2b-4493-4297-951d-f97e62a33f95 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:20.537724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3fb2b-4493-4297-951d-f97e62a33f95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:20.538003 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf3fb2b-4493-4297-951d-f97e62a33f95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:20.547340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3fb2b-4493-4297-951d-f97e62a33f95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:26:20.548765 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf3fb2b-4493-4297-951d-f97e62a33f95 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:26:20.558762 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 760/1533] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:26:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:26:23.989402 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778e42b-62b2-4846-9eb8-3b047645dc08 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:23.989983 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778e42b-62b2-4846-9eb8-3b047645dc08 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:23.989983 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778e42b-62b2-4846-9eb8-3b047645dc08 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:23.991383 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 774/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 22:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:26:23.996780 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3758aec-b1b2-4ab0-81de-4c3ad6cb51e0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:23.997186 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3758aec-b1b2-4ab0-81de-4c3ad6cb51e0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:23.997325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3758aec-b1b2-4ab0-81de-4c3ad6cb51e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:24.005089 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=81 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:26:24.018046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d3758aec-b1b2-4ab0-81de-4c3ad6cb51e0 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:26:24.019231 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d3758aec-b1b2-4ab0-81de-4c3ad6cb51e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:26:24.027552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 761/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 22:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:26:24.366901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778e42b-62b2-4846-9eb8-3b047645dc08 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:26:24.368897 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7778e42b-62b2-4846-9eb8-3b047645dc08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:26:24.439381 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:26:24.453650 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778e42b-62b2-4846-9eb8-3b047645dc08 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:26:24.457253 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b2d63f-92d5-4f21-aa1d-f5987cc7d75e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:26:24.457912 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2d63f-92d5-4f21-aa1d-f5987cc7d75e service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:24.458198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2d63f-92d5-4f21-aa1d-f5987cc7d75e service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:24.458479 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2d63f-92d5-4f21-aa1d-f5987cc7d75e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:24.468496 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2d63f-92d5-4f21-aa1d-f5987cc7d75e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:26:24.469963 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b2d63f-92d5-4f21-aa1d-f5987cc7d75e service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:26:24.479223 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 775/1536] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:26:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:26:26.559834 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d8033-6cc3-44ae-8543-bc34c8567e08 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:26.560344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d8033-6cc3-44ae-8543-bc34c8567e08 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:26.560344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d8033-6cc3-44ae-8543-bc34c8567e08 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:26.561658 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 762/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 28 22:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:26:26.566705 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359a4f2-55fe-4afb-8956-da0553a5180a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:26.567114 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359a4f2-55fe-4afb-8956-da0553a5180a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:26.567243 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359a4f2-55fe-4afb-8956-da0553a5180a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:26.586719 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6359a4f2-55fe-4afb-8956-da0553a5180a None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:26:26.587927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6359a4f2-55fe-4afb-8956-da0553a5180a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:26:26.595706 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 776/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 28 22:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:26:26.905796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d8033-6cc3-44ae-8543-bc34c8567e08 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:26:26.907456 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5d0d8033-6cc3-44ae-8543-bc34c8567e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:26:26.920773 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d8033-6cc3-44ae-8543-bc34c8567e08 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:26:26.924229 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31abf8a-437a-4938-a094-8e6f0f243487 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:26:26.924859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31abf8a-437a-4938-a094-8e6f0f243487 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:26:26.925141 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31abf8a-437a-4938-a094-8e6f0f243487 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:26:26.925418 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31abf8a-437a-4938-a094-8e6f0f243487 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:26:26.935375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31abf8a-437a-4938-a094-8e6f0f243487 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:26:26.936791 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31abf8a-437a-4938-a094-8e6f0f243487 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:26:26.947436 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 763/1539] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:26:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:26:36.581861 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=11 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:26:36.942059 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=8 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:27:57.099488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fab6b7-01ae-4fda-a4af-fe7d8cd07850 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.100613 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fab6b7-01ae-4fda-a4af-fe7d8cd07850 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.100613 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fab6b7-01ae-4fda-a4af-fe7d8cd07850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.129705 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-96fab6b7-01ae-4fda-a4af-fe7d8cd07850 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:57.131412 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-96fab6b7-01ae-4fda-a4af-fe7d8cd07850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:57.156438 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 777/1540] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:27:57.165888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be84a0-6801-49e1-acf4-971740706025 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.166447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be84a0-6801-49e1-acf4-971740706025 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.166447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be84a0-6801-49e1-acf4-971740706025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.173275 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086689ee-6aee-455d-864f-33e71375130f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.173597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086689ee-6aee-455d-864f-33e71375130f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.173949 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086689ee-6aee-455d-864f-33e71375130f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.195094 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e7be84a0-6801-49e1-acf4-971740706025 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:57.196302 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e7be84a0-6801-49e1-acf4-971740706025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:57.203549 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-086689ee-6aee-455d-864f-33e71375130f None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:57.204786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-086689ee-6aee-455d-864f-33e71375130f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:57.209042 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 764/1541] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:27:57.213857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be84a0-6801-49e1-acf4-971740706025 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:57.221396 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 778/1542] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:27:57.228973 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be84a0-6801-49e1-acf4-971740706025 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:57.230489 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a216cd-3faa-4550-a261-1246ab949d2b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.230851 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a216cd-3faa-4550-a261-1246ab949d2b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.231130 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a216cd-3faa-4550-a261-1246ab949d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.243669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3214d10-dcb7-4ab4-bcde-08ea022840d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:57.244574 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3214d10-dcb7-4ab4-bcde-08ea022840d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.244864 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3214d10-dcb7-4ab4-bcde-08ea022840d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.245137 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3214d10-dcb7-4ab4-bcde-08ea022840d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.245884 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3214d10-dcb7-4ab4-bcde-08ea022840d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:27:57.246192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3214d10-dcb7-4ab4-bcde-08ea022840d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:57.247227 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-b3214d10-dcb7-4ab4-bcde-08ea022840d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 28 22:27:57.247227 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jul 28 22:27:57.247227 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: Jul 28 22:27:57.248348 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3214d10-dcb7-4ab4-bcde-08ea022840d9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:57.274331 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-06a216cd-3faa-4550-a261-1246ab949d2b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:57.275589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-06a216cd-3faa-4550-a261-1246ab949d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:57.277607 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 765/1543] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:27:57.290759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 779/1544] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:27:57.296378 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3214d10-dcb7-4ab4-bcde-08ea022840d9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:57.297908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a216cd-3faa-4550-a261-1246ab949d2b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:57.312990 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a216cd-3faa-4550-a261-1246ab949d2b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:57.318862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19c918-69f3-4877-b73c-66dcaf735ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:57.319529 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19c918-69f3-4877-b73c-66dcaf735ab9 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.319815 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19c918-69f3-4877-b73c-66dcaf735ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.320099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19c918-69f3-4877-b73c-66dcaf735ab9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.320797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c19c918-69f3-4877-b73c-66dcaf735ab9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:57.323931 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3214d10-dcb7-4ab4-bcde-08ea022840d9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:57.330419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-8c19c918-69f3-4877-b73c-66dcaf735ab9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 22:27:57.330419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Jul 28 22:27:57.330419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: Jul 28 22:27:57.331743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c19c918-69f3-4877-b73c-66dcaf735ab9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:57.342276 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e256b-57cc-415c-be42-e4d10b8f758d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:57.343670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e256b-57cc-415c-be42-e4d10b8f758d admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.343966 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e256b-57cc-415c-be42-e4d10b8f758d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.344248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e256b-57cc-415c-be42-e4d10b8f758d admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.344964 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e256b-57cc-415c-be42-e4d10b8f758d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:27:57.345274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e256b-57cc-415c-be42-e4d10b8f758d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:57.347463 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e256b-57cc-415c-be42-e4d10b8f758d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:57.378838 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 766/1545] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:27:57.388105 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e256b-57cc-415c-be42-e4d10b8f758d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:57.410873 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 780/1546] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:27:57 2022] POST /identity/v3/projects => generated 375 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:27:57.415384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19c918-69f3-4877-b73c-66dcaf735ab9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:57.416214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e256b-57cc-415c-be42-e4d10b8f758d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:57.571053 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45bc170-e953-4e85-9389-2d97f1673e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:57.571688 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bc170-e953-4e85-9389-2d97f1673e8f admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.571970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bc170-e953-4e85-9389-2d97f1673e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.572245 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45bc170-e953-4e85-9389-2d97f1673e8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.572921 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45bc170-e953-4e85-9389-2d97f1673e8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:57.576634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19c918-69f3-4877-b73c-66dcaf735ab9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:57.578073 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45bc170-e953-4e85-9389-2d97f1673e8f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:57.592962 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ede9ae-32b1-4271-b9ff-c59d28df4a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:57.593630 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ede9ae-32b1-4271-b9ff-c59d28df4a99 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.593914 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ede9ae-32b1-4271-b9ff-c59d28df4a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.594188 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ede9ae-32b1-4271-b9ff-c59d28df4a99 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.594870 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ede9ae-32b1-4271-b9ff-c59d28df4a99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:57.597056 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ede9ae-32b1-4271-b9ff-c59d28df4a99 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:57.648004 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 781/1547] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:27:57 2022] POST /identity/v3/users => generated 288 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:27:57.655062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d8f71-d1fe-4f30-a81e-e5b9bc4be6e3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.655382 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d8f71-d1fe-4f30-a81e-e5b9bc4be6e3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.655648 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d8f71-d1fe-4f30-a81e-e5b9bc4be6e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.664174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 767/1548] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:27:57 2022] POST /identity/v3/projects => generated 383 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:27:57.665636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996d5f2-a939-4c6e-ada6-821b6044a82c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.665929 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996d5f2-a939-4c6e-ada6-821b6044a82c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.666210 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996d5f2-a939-4c6e-ada6-821b6044a82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.687745 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e45d8f71-d1fe-4f30-a81e-e5b9bc4be6e3 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:57.688981 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e45d8f71-d1fe-4f30-a81e-e5b9bc4be6e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:57.707917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b996d5f2-a939-4c6e-ada6-821b6044a82c None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:57.709188 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b996d5f2-a939-4c6e-ada6-821b6044a82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:57.770206 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 782/1549] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:27:57.770206 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d8f71-d1fe-4f30-a81e-e5b9bc4be6e3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:57.782443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d8f71-d1fe-4f30-a81e-e5b9bc4be6e3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:57.783939 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 768/1550] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:27:57.784912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996d5f2-a939-4c6e-ada6-821b6044a82c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:57.788598 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63924454-ae7a-43f2-9ca6-28463b48827c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:57.789227 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63924454-ae7a-43f2-9ca6-28463b48827c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.789516 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63924454-ae7a-43f2-9ca6-28463b48827c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.789825 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63924454-ae7a-43f2-9ca6-28463b48827c admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.790855 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63924454-ae7a-43f2-9ca6-28463b48827c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:57.793089 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63924454-ae7a-43f2-9ca6-28463b48827c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:57.795026 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996d5f2-a939-4c6e-ada6-821b6044a82c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:57.799634 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba423550-eec1-4f16-a364-cf39eba960cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:57.800239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba423550-eec1-4f16-a364-cf39eba960cb admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.800523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba423550-eec1-4f16-a364-cf39eba960cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.800798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba423550-eec1-4f16-a364-cf39eba960cb admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.801479 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba423550-eec1-4f16-a364-cf39eba960cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:57.802554 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-ba423550-eec1-4f16-a364-cf39eba960cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 22:27:57.802554 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jul 28 22:27:57.802554 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: Jul 28 22:27:57.803359 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 783/1551] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:27:57.803859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba423550-eec1-4f16-a364-cf39eba960cb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:57.804568 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973bb39-1b9b-422f-8b08-562989cd56b5 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.804863 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973bb39-1b9b-422f-8b08-562989cd56b5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.805140 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6973bb39-1b9b-422f-8b08-562989cd56b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.824290 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6973bb39-1b9b-422f-8b08-562989cd56b5 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:57.825506 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6973bb39-1b9b-422f-8b08-562989cd56b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:57.825652 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 769/1552] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:27:57 2022] POST /identity/v3/users => generated 292 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:27:57.827015 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71f676-8243-46c2-9bd7-816baba78988 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.827309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71f676-8243-46c2-9bd7-816baba78988 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.827649 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec71f676-8243-46c2-9bd7-816baba78988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.834383 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 784/1553] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:27:57.835813 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea1c19-03c6-4ca3-889d-860fd7dc8795 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.835966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea1c19-03c6-4ca3-889d-860fd7dc8795 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.836207 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea1c19-03c6-4ca3-889d-860fd7dc8795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.851027 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ec71f676-8243-46c2-9bd7-816baba78988 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:57.852293 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ec71f676-8243-46c2-9bd7-816baba78988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:57.856790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a3ea1c19-03c6-4ca3-889d-860fd7dc8795 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:57.858038 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a3ea1c19-03c6-4ca3-889d-860fd7dc8795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:57.862353 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 770/1554] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:27:57.863280 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504cc84-0b3d-4383-89e0-4af1b6d56c1c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.863561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504cc84-0b3d-4383-89e0-4af1b6d56c1c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.863839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8504cc84-0b3d-4383-89e0-4af1b6d56c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.873313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 785/1555] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:27:57.874748 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519a378-228d-4069-909f-dc8af25c03bf None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.875047 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519a378-228d-4069-909f-dc8af25c03bf None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.875326 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a519a378-228d-4069-909f-dc8af25c03bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.883911 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8504cc84-0b3d-4383-89e0-4af1b6d56c1c None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:57.885399 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8504cc84-0b3d-4383-89e0-4af1b6d56c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:57.893508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 771/1556] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:27:57.894811 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b49b33-a949-47ed-924c-4e292e089419 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.895104 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b49b33-a949-47ed-924c-4e292e089419 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.895386 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b49b33-a949-47ed-924c-4e292e089419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.902167 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a519a378-228d-4069-909f-dc8af25c03bf None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:57.903315 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a519a378-228d-4069-909f-dc8af25c03bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:57.912346 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 786/1557] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:27:57.913193 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519a378-228d-4069-909f-dc8af25c03bf None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:57.915830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-67b49b33-a949-47ed-924c-4e292e089419 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:57.917259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-67b49b33-a949-47ed-924c-4e292e089419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:57.925111 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 772/1558] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:27:57.925930 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b49b33-a949-47ed-924c-4e292e089419 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:57.926645 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a519a378-228d-4069-909f-dc8af25c03bf None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:57.933790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3a369-8e65-446a-94a4-b43b36f99017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:57.934226 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3a369-8e65-446a-94a4-b43b36f99017 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.934438 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3a369-8e65-446a-94a4-b43b36f99017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.934715 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3a369-8e65-446a-94a4-b43b36f99017 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.935419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3a369-8e65-446a-94a4-b43b36f99017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:57.936533 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b49b33-a949-47ed-924c-4e292e089419 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:57.937080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3a369-8e65-446a-94a4-b43b36f99017 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:57.940019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:27:57.941411 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e7761-418d-40b0-9a47-3096d26cf232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:57.942046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e7761-418d-40b0-9a47-3096d26cf232 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.942612 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e7761-418d-40b0-9a47-3096d26cf232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.942992 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e7761-418d-40b0-9a47-3096d26cf232 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.943758 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e7761-418d-40b0-9a47-3096d26cf232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:57.945749 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e7761-418d-40b0-9a47-3096d26cf232 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:57.952823 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 787/1559] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:27:57.954160 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2008d-22de-455f-8981-f1e6d228d338 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.954456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2008d-22de-455f-8981-f1e6d228d338 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.954738 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc2008d-22de-455f-8981-f1e6d228d338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.959424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 773/1560] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:27:57.959424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9722346-b09f-489e-84ca-895729d6f952 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.959424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9722346-b09f-489e-84ca-895729d6f952 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.959424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9722346-b09f-489e-84ca-895729d6f952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.966532 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:27:57.977050 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-4fc2008d-22de-455f-8981-f1e6d228d338 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:57.978359 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4fc2008d-22de-455f-8981-f1e6d228d338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:57.979156 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a9722346-b09f-489e-84ca-895729d6f952 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:57.980306 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a9722346-b09f-489e-84ca-895729d6f952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:57.986314 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 788/1561] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:27:57.987300 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c8cd3-7855-453a-a61e-848a9fcb7c0c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.987587 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c8cd3-7855-453a-a61e-848a9fcb7c0c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.987865 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c8cd3-7855-453a-a61e-848a9fcb7c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:57.991678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 774/1562] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:27:57.992858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eab615-cf4f-4031-980a-961713164364 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:57.993147 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eab615-cf4f-4031-980a-961713164364 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:57.993414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eab615-cf4f-4031-980a-961713164364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.007918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b73c8cd3-7855-453a-a61e-848a9fcb7c0c None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:58.009117 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b73c8cd3-7855-453a-a61e-848a9fcb7c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:58.012308 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e3eab615-cf4f-4031-980a-961713164364 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:58.013450 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e3eab615-cf4f-4031-980a-961713164364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:58.017377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 789/1563] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:27:58.018171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c8cd3-7855-453a-a61e-848a9fcb7c0c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.021702 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 775/1564] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:27:58.022473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eab615-cf4f-4031-980a-961713164364 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.031322 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c8cd3-7855-453a-a61e-848a9fcb7c0c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.031712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eab615-cf4f-4031-980a-961713164364 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.035924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbfc0b-b9bd-4f72-9415-a2d3bf04d1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.036146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73620381-01e2-41d4-9877-43ab94fe03da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.037042 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbfc0b-b9bd-4f72-9415-a2d3bf04d1bb admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.037248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73620381-01e2-41d4-9877-43ab94fe03da admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.037416 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbfc0b-b9bd-4f72-9415-a2d3bf04d1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.037606 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73620381-01e2-41d4-9877-43ab94fe03da admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.037704 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dbfc0b-b9bd-4f72-9415-a2d3bf04d1bb admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.037819 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73620381-01e2-41d4-9877-43ab94fe03da admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.038398 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dbfc0b-b9bd-4f72-9415-a2d3bf04d1bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:27:58.038512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73620381-01e2-41d4-9877-43ab94fe03da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:27:58.038701 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dbfc0b-b9bd-4f72-9415-a2d3bf04d1bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.038801 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73620381-01e2-41d4-9877-43ab94fe03da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.040296 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dbfc0b-b9bd-4f72-9415-a2d3bf04d1bb admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.040405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73620381-01e2-41d4-9877-43ab94fe03da admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.052779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 776/1565] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:27:58.053234 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 790/1566] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:27:58.053678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73620381-01e2-41d4-9877-43ab94fe03da admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.054347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbfc0b-b9bd-4f72-9415-a2d3bf04d1bb admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.064676 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dbfc0b-b9bd-4f72-9415-a2d3bf04d1bb admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.069516 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c85ac-0b6b-4515-b551-90a0a0e584c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.070262 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c85ac-0b6b-4515-b551-90a0a0e584c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.070545 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c85ac-0b6b-4515-b551-90a0a0e584c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.070821 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c85ac-0b6b-4515-b551-90a0a0e584c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.071125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73620381-01e2-41d4-9877-43ab94fe03da admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.071515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c85ac-0b6b-4515-b551-90a0a0e584c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.073110 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55c85ac-0b6b-4515-b551-90a0a0e584c8 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.075832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33e890-c579-4229-9cc3-29fabd09d034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.076502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33e890-c579-4229-9cc3-29fabd09d034 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.076787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33e890-c579-4229-9cc3-29fabd09d034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.077065 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33e890-c579-4229-9cc3-29fabd09d034 admin admin] PATH_INFO: `/v3/projects/d829cbd8678145668ea811e01093fcfc/users/7476eccb0a6f4195936bec75f7f80c7d/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.084599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 791/1567] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:27:58.085289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c85ac-0b6b-4515-b551-90a0a0e584c8 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.092332 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33e890-c579-4229-9cc3-29fabd09d034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d829cbd8678145668ea811e01093fcfc, user_id=7476eccb0a6f4195936bec75f7f80c7d, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.095051 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33e890-c579-4229-9cc3-29fabd09d034 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.095452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55c85ac-0b6b-4515-b551-90a0a0e584c8 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.101552 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51212c53-5f10-4ea7-8ceb-d93b6d57e9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.102241 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51212c53-5f10-4ea7-8ceb-d93b6d57e9b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.102447 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51212c53-5f10-4ea7-8ceb-d93b6d57e9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.102712 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51212c53-5f10-4ea7-8ceb-d93b6d57e9b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.103463 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51212c53-5f10-4ea7-8ceb-d93b6d57e9b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:27:58.103787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51212c53-5f10-4ea7-8ceb-d93b6d57e9b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.105407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51212c53-5f10-4ea7-8ceb-d93b6d57e9b3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.109568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 777/1568] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:27:58 2022] PUT /identity/v3/projects/d829cbd8678145668ea811e01093fcfc/users/7476eccb0a6f4195936bec75f7f80c7d/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:27:58.110433 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33e890-c579-4229-9cc3-29fabd09d034 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.118329 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 792/1569] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:27:58.119078 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51212c53-5f10-4ea7-8ceb-d93b6d57e9b3 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.130048 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51212c53-5f10-4ea7-8ceb-d93b6d57e9b3 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.175474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33e890-c579-4229-9cc3-29fabd09d034 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.180401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b10e6-3b39-4463-981b-8cadf9b0411b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.181025 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b10e6-3b39-4463-981b-8cadf9b0411b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.181309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b10e6-3b39-4463-981b-8cadf9b0411b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.181612 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b10e6-3b39-4463-981b-8cadf9b0411b admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.182317 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b10e6-3b39-4463-981b-8cadf9b0411b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:27:58.182625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b10e6-3b39-4463-981b-8cadf9b0411b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.184231 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b10e6-3b39-4463-981b-8cadf9b0411b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.190097 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88276f-cb21-4f89-8dfd-aa0d88496b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.190718 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88276f-cb21-4f89-8dfd-aa0d88496b48 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.191489 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88276f-cb21-4f89-8dfd-aa0d88496b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.191772 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88276f-cb21-4f89-8dfd-aa0d88496b48 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.192995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88276f-cb21-4f89-8dfd-aa0d88496b48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:27:58.193970 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88276f-cb21-4f89-8dfd-aa0d88496b48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.195631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88276f-cb21-4f89-8dfd-aa0d88496b48 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.196491 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 778/1570] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:27:58.197325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b10e6-3b39-4463-981b-8cadf9b0411b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.208061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 793/1571] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:27:58.209433 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88276f-cb21-4f89-8dfd-aa0d88496b48 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.209869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b10e6-3b39-4463-981b-8cadf9b0411b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.214842 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037854d5-5b54-4859-aee6-1accfef8fd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.215455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037854d5-5b54-4859-aee6-1accfef8fd92 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.215742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037854d5-5b54-4859-aee6-1accfef8fd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.216012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037854d5-5b54-4859-aee6-1accfef8fd92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.216713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037854d5-5b54-4859-aee6-1accfef8fd92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:27:58.217208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037854d5-5b54-4859-aee6-1accfef8fd92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.218667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037854d5-5b54-4859-aee6-1accfef8fd92 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.221851 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88276f-cb21-4f89-8dfd-aa0d88496b48 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.229201 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53101fac-8d4a-4875-ad13-937c77a315a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.229876 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53101fac-8d4a-4875-ad13-937c77a315a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.230154 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53101fac-8d4a-4875-ad13-937c77a315a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.230428 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53101fac-8d4a-4875-ad13-937c77a315a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.231102 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53101fac-8d4a-4875-ad13-937c77a315a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.231753 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 779/1572] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:27:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:27:58.232581 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037854d5-5b54-4859-aee6-1accfef8fd92 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.233056 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53101fac-8d4a-4875-ad13-937c77a315a7 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.243947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037854d5-5b54-4859-aee6-1accfef8fd92 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.248864 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738120f1-0186-4e5f-a66a-381f9af92859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.249501 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738120f1-0186-4e5f-a66a-381f9af92859 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.249810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738120f1-0186-4e5f-a66a-381f9af92859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.250092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738120f1-0186-4e5f-a66a-381f9af92859 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.250751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738120f1-0186-4e5f-a66a-381f9af92859 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.252824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738120f1-0186-4e5f-a66a-381f9af92859 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.267041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 794/1573] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:27:58 2022] POST /identity/v3/projects => generated 389 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:27:58.267041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53101fac-8d4a-4875-ad13-937c77a315a7 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.287941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 780/1574] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:27:58 2022] POST /identity/v3/projects => generated 403 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:27:58.287941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285e526-c214-4107-b45b-13a1462ac21d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.287941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285e526-c214-4107-b45b-13a1462ac21d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.287941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285e526-c214-4107-b45b-13a1462ac21d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.304670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e285e526-c214-4107-b45b-13a1462ac21d None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:58.305925 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e285e526-c214-4107-b45b-13a1462ac21d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:58.328357 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53101fac-8d4a-4875-ad13-937c77a315a7 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.352158 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:27:58.367048 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 781/1575] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:27:58.368193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285e526-c214-4107-b45b-13a1462ac21d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.377618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285e526-c214-4107-b45b-13a1462ac21d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.383061 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191ceb7-7a94-45f2-8fbe-5142dcf2e2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.383669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191ceb7-7a94-45f2-8fbe-5142dcf2e2c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.383952 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191ceb7-7a94-45f2-8fbe-5142dcf2e2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.384495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9191ceb7-7a94-45f2-8fbe-5142dcf2e2c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.385152 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191ceb7-7a94-45f2-8fbe-5142dcf2e2c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.387151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9191ceb7-7a94-45f2-8fbe-5142dcf2e2c9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.389756 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89541c-7f4c-44b9-82df-7b0602ccb81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.390423 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89541c-7f4c-44b9-82df-7b0602ccb81c admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.390704 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89541c-7f4c-44b9-82df-7b0602ccb81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.390978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89541c-7f4c-44b9-82df-7b0602ccb81c admin admin] PATH_INFO: `/v3/projects/d0c7b146e08f4a9ea11f1a73099eb7ea/users/d38b517910894f7fa4351a2120b466d7/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.398301 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:27:58.406708 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89541c-7f4c-44b9-82df-7b0602ccb81c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0c7b146e08f4a9ea11f1a73099eb7ea, user_id=d38b517910894f7fa4351a2120b466d7, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.407906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-1c89541c-7f4c-44b9-82df-7b0602ccb81c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 22:27:58.407906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Jul 28 22:27:58.407906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: Jul 28 22:27:58.410292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89541c-7f4c-44b9-82df-7b0602ccb81c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.418190 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 782/1576] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:27:58 2022] POST /identity/v3/projects => generated 391 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:27:58.419402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191ceb7-7a94-45f2-8fbe-5142dcf2e2c9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.427881 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 795/1577] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:27:58 2022] PUT /identity/v3/projects/d0c7b146e08f4a9ea11f1a73099eb7ea/users/d38b517910894f7fa4351a2120b466d7/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:27:58.428755 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89541c-7f4c-44b9-82df-7b0602ccb81c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.484316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9191ceb7-7a94-45f2-8fbe-5142dcf2e2c9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.488958 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943c2df-0ae8-4407-b893-c25856523f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.489116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89541c-7f4c-44b9-82df-7b0602ccb81c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.490235 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943c2df-0ae8-4407-b893-c25856523f83 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.490523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943c2df-0ae8-4407-b893-c25856523f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.490800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0943c2df-0ae8-4407-b893-c25856523f83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.491472 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943c2df-0ae8-4407-b893-c25856523f83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.493384 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0943c2df-0ae8-4407-b893-c25856523f83 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.494899 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4debdc-93bc-4440-bb92-fcd5ea1057bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.495617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4debdc-93bc-4440-bb92-fcd5ea1057bb admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.495903 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4debdc-93bc-4440-bb92-fcd5ea1057bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.496176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4debdc-93bc-4440-bb92-fcd5ea1057bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.496830 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4debdc-93bc-4440-bb92-fcd5ea1057bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.498748 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4debdc-93bc-4440-bb92-fcd5ea1057bb admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.521131 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 783/1578] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:27:58 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:27:58.522071 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943c2df-0ae8-4407-b893-c25856523f83 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.526716 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 796/1579] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:27:58 2022] POST /identity/v3/projects => generated 379 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:27:58.527410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4debdc-93bc-4440-bb92-fcd5ea1057bb admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.591644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0943c2df-0ae8-4407-b893-c25856523f83 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.592284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4debdc-93bc-4440-bb92-fcd5ea1057bb admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.596208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7dbcbc-62fa-42c7-b6bd-6e40df7f8ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.596825 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7dbcbc-62fa-42c7-b6bd-6e40df7f8ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.596945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52849cb7-7ecb-46a2-8993-3578dafc62e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.597108 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7dbcbc-62fa-42c7-b6bd-6e40df7f8ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.597379 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7dbcbc-62fa-42c7-b6bd-6e40df7f8ee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.597540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52849cb7-7ecb-46a2-8993-3578dafc62e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.597834 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52849cb7-7ecb-46a2-8993-3578dafc62e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.598072 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7dbcbc-62fa-42c7-b6bd-6e40df7f8ee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.598180 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52849cb7-7ecb-46a2-8993-3578dafc62e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.599336 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52849cb7-7ecb-46a2-8993-3578dafc62e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.601095 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7dbcbc-62fa-42c7-b6bd-6e40df7f8ee3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.601263 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52849cb7-7ecb-46a2-8993-3578dafc62e6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.623355 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 797/1580] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:27:58 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:27:58.624555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52849cb7-7ecb-46a2-8993-3578dafc62e6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.629009 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 784/1581] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:27:58 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:27:58.629871 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7dbcbc-62fa-42c7-b6bd-6e40df7f8ee3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.636729 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52849cb7-7ecb-46a2-8993-3578dafc62e6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.658139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:27:58.688964 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7dbcbc-62fa-42c7-b6bd-6e40df7f8ee3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.692591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dde2a9-94a3-4dd4-b0f6-6252b7cedb21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.693207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dde2a9-94a3-4dd4-b0f6-6252b7cedb21 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.693796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dde2a9-94a3-4dd4-b0f6-6252b7cedb21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.693796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dde2a9-94a3-4dd4-b0f6-6252b7cedb21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.700262 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f64bc-005e-4323-bd7e-787594b8cff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.705869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f64bc-005e-4323-bd7e-787594b8cff6 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.706254 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f64bc-005e-4323-bd7e-787594b8cff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.706766 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f64bc-005e-4323-bd7e-787594b8cff6 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.707531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:27:58.708197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f64bc-005e-4323-bd7e-787594b8cff6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.710624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f64bc-005e-4323-bd7e-787594b8cff6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.738781 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 798/1582] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:27:58 2022] POST /identity/v3/users => generated 302 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:27:58.738781 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f64bc-005e-4323-bd7e-787594b8cff6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.747998 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f64bc-005e-4323-bd7e-787594b8cff6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.753035 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41ab70-f42f-4a8a-9698-9d1e041bafcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.753740 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41ab70-f42f-4a8a-9698-9d1e041bafcb admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.754095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41ab70-f42f-4a8a-9698-9d1e041bafcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.754469 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b41ab70-f42f-4a8a-9698-9d1e041bafcb admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.755328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b41ab70-f42f-4a8a-9698-9d1e041bafcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.757327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b41ab70-f42f-4a8a-9698-9d1e041bafcb admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.759877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dde2a9-94a3-4dd4-b0f6-6252b7cedb21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.761403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dde2a9-94a3-4dd4-b0f6-6252b7cedb21 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.772000 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 785/1583] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:27:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:27:58.773412 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da20a1-3689-48b5-a1dc-7158581fd4ab None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.773838 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da20a1-3689-48b5-a1dc-7158581fd4ab None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.774106 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38da20a1-3689-48b5-a1dc-7158581fd4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.779896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 799/1584] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:27:58 2022] POST /identity/v3/users => generated 296 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:27:58.780756 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41ab70-f42f-4a8a-9698-9d1e041bafcb admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.792074 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b41ab70-f42f-4a8a-9698-9d1e041bafcb admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.797447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-38da20a1-3689-48b5-a1dc-7158581fd4ab None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:58.797757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f374f-0b7a-4750-8f7c-e80fb259ab7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.798518 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f374f-0b7a-4750-8f7c-e80fb259ab7d admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.798697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-38da20a1-3689-48b5-a1dc-7158581fd4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:58.798886 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f374f-0b7a-4750-8f7c-e80fb259ab7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.799182 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f374f-0b7a-4750-8f7c-e80fb259ab7d admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.799948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f374f-0b7a-4750-8f7c-e80fb259ab7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.802670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f374f-0b7a-4750-8f7c-e80fb259ab7d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.810875 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 786/1585] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:27:58.810875 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da20a1-3689-48b5-a1dc-7158581fd4ab None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.823036 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38da20a1-3689-48b5-a1dc-7158581fd4ab None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.828154 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091b410-dd65-4316-bbd0-8946221cbb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.831583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091b410-dd65-4316-bbd0-8946221cbb34 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.831583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091b410-dd65-4316-bbd0-8946221cbb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.831834 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 800/1586] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:27:58 2022] POST /identity/v3/users => generated 283 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:27:58.831941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091b410-dd65-4316-bbd0-8946221cbb34 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.831941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7091b410-dd65-4316-bbd0-8946221cbb34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.832114 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f374f-0b7a-4750-8f7c-e80fb259ab7d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.832288 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7091b410-dd65-4316-bbd0-8946221cbb34 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.844440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f374f-0b7a-4750-8f7c-e80fb259ab7d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.851245 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57eec32-99cb-4122-9c4c-8c26efe84d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.852172 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57eec32-99cb-4122-9c4c-8c26efe84d11 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.852603 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57eec32-99cb-4122-9c4c-8c26efe84d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.853055 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57eec32-99cb-4122-9c4c-8c26efe84d11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.854091 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57eec32-99cb-4122-9c4c-8c26efe84d11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.856266 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57eec32-99cb-4122-9c4c-8c26efe84d11 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.856962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 787/1587] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:27:58 2022] POST /identity/v3/users => generated 290 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:27:58.857954 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091b410-dd65-4316-bbd0-8946221cbb34 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.869508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091b410-dd65-4316-bbd0-8946221cbb34 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.870380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 801/1588] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:27:58.871446 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57eec32-99cb-4122-9c4c-8c26efe84d11 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.878421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf4589-0584-40d3-8a1a-c366e96366cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.879610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf4589-0584-40d3-8a1a-c366e96366cc admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.879985 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf4589-0584-40d3-8a1a-c366e96366cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.880410 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf4589-0584-40d3-8a1a-c366e96366cc admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.881420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf4589-0584-40d3-8a1a-c366e96366cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.883507 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf4589-0584-40d3-8a1a-c366e96366cc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.883918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57eec32-99cb-4122-9c4c-8c26efe84d11 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.889401 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd3fab-9da9-4cd3-ab65-29650191b0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.890214 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd3fab-9da9-4cd3-ab65-29650191b0ab admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.890645 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd3fab-9da9-4cd3-ab65-29650191b0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.891066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd3fab-9da9-4cd3-ab65-29650191b0ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.892018 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd3fab-9da9-4cd3-ab65-29650191b0ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.893955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd3fab-9da9-4cd3-ab65-29650191b0ab admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.905667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 802/1589] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:27:58.906604 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd3fab-9da9-4cd3-ab65-29650191b0ab admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.910447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 788/1590] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:27:58 2022] POST /identity/v3/users => generated 293 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:27:58.910909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf4589-0584-40d3-8a1a-c366e96366cc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.913310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0bcf4589-0584-40d3-8a1a-c366e96366cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:58.924178 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd3fab-9da9-4cd3-ab65-29650191b0ab admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.926018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf4589-0584-40d3-8a1a-c366e96366cc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.929330 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b226c3-82f7-4eb5-a605-18fb7b9a8861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.930131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b226c3-82f7-4eb5-a605-18fb7b9a8861 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.930415 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b226c3-82f7-4eb5-a605-18fb7b9a8861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.930528 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a8aa2-17c8-4371-bd1c-5b287264b8a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.930737 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b226c3-82f7-4eb5-a605-18fb7b9a8861 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.931066 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a8aa2-17c8-4371-bd1c-5b287264b8a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.931346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a8aa2-17c8-4371-bd1c-5b287264b8a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.931460 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b226c3-82f7-4eb5-a605-18fb7b9a8861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.931641 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a8aa2-17c8-4371-bd1c-5b287264b8a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.933221 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b226c3-82f7-4eb5-a605-18fb7b9a8861 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.941795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a8aa2-17c8-4371-bd1c-5b287264b8a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.944713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a8aa2-17c8-4371-bd1c-5b287264b8a0 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.950453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 803/1591] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:27:58.950453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b226c3-82f7-4eb5-a605-18fb7b9a8861 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.955904 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 789/1592] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:27:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:27:58.966688 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a8aa2-17c8-4371-bd1c-5b287264b8a0 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:58.979388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b226c3-82f7-4eb5-a605-18fb7b9a8861 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.981084 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a8aa2-17c8-4371-bd1c-5b287264b8a0 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:58.989186 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06794eb-9b8b-4d37-84b7-a0d9ed5b2719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.991227 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06794eb-9b8b-4d37-84b7-a0d9ed5b2719 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.991514 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06794eb-9b8b-4d37-84b7-a0d9ed5b2719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.992059 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06794eb-9b8b-4d37-84b7-a0d9ed5b2719 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:58.992778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06794eb-9b8b-4d37-84b7-a0d9ed5b2719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:58.994635 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06794eb-9b8b-4d37-84b7-a0d9ed5b2719 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:58.995442 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adef9a-ef82-4873-8927-13661d15e176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:58.998775 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adef9a-ef82-4873-8927-13661d15e176 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:58.999311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adef9a-ef82-4873-8927-13661d15e176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:58.999740 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adef9a-ef82-4873-8927-13661d15e176 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:59.003447 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2adef9a-ef82-4873-8927-13661d15e176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:59.006007 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2adef9a-ef82-4873-8927-13661d15e176 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:59.012433 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 790/1593] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:27:59.014541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06794eb-9b8b-4d37-84b7-a0d9ed5b2719 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:59.023498 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 804/1594] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:27:58 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:27:59.024377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adef9a-ef82-4873-8927-13661d15e176 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:59.028187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06794eb-9b8b-4d37-84b7-a0d9ed5b2719 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:59.034437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e10ad-af88-43ee-822e-67680df765fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:59.035182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e10ad-af88-43ee-822e-67680df765fd admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:59.035443 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e10ad-af88-43ee-822e-67680df765fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:59.035793 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38e10ad-af88-43ee-822e-67680df765fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:59.036977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e10ad-af88-43ee-822e-67680df765fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:59.038688 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38e10ad-af88-43ee-822e-67680df765fd admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:59.047123 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adef9a-ef82-4873-8927-13661d15e176 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:59.055191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 791/1595] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:27:59.055969 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a274140-1c17-49c2-8f48-656716b44f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:59.056112 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e10ad-af88-43ee-822e-67680df765fd admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:59.056779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a274140-1c17-49c2-8f48-656716b44f8e admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:59.057252 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a274140-1c17-49c2-8f48-656716b44f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:59.057758 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a274140-1c17-49c2-8f48-656716b44f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:59.059092 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a274140-1c17-49c2-8f48-656716b44f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:59.061635 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a274140-1c17-49c2-8f48-656716b44f8e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:59.067301 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38e10ad-af88-43ee-822e-67680df765fd admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:59.074287 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2766b-e450-46d7-8885-51df6497cc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:59.075116 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2766b-e450-46d7-8885-51df6497cc5b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:59.075404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2766b-e450-46d7-8885-51df6497cc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:59.075752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2766b-e450-46d7-8885-51df6497cc5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:59.076667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b2766b-e450-46d7-8885-51df6497cc5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:59.078374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b2766b-e450-46d7-8885-51df6497cc5b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:59.083206 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 805/1596] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:27:59.085978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a274140-1c17-49c2-8f48-656716b44f8e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:59.093497 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 792/1597] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:27:59.094819 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2766b-e450-46d7-8885-51df6497cc5b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:59.097830 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a274140-1c17-49c2-8f48-656716b44f8e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:59.103733 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799ce94-576d-4e9f-85b9-8ab21d24f69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:59.105285 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799ce94-576d-4e9f-85b9-8ab21d24f69d admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:59.105640 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799ce94-576d-4e9f-85b9-8ab21d24f69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:59.105935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799ce94-576d-4e9f-85b9-8ab21d24f69d admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:59.106911 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799ce94-576d-4e9f-85b9-8ab21d24f69d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:59.108648 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b2766b-e450-46d7-8885-51df6497cc5b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:59.109371 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799ce94-576d-4e9f-85b9-8ab21d24f69d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:59.115464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10845d-1763-4ce8-a0e1-6a36ec06951d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:59.116617 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10845d-1763-4ce8-a0e1-6a36ec06951d admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:59.117174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10845d-1763-4ce8-a0e1-6a36ec06951d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:59.117685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10845d-1763-4ce8-a0e1-6a36ec06951d admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:59.118747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10845d-1763-4ce8-a0e1-6a36ec06951d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:59.120785 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10845d-1763-4ce8-a0e1-6a36ec06951d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:59.125401 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 806/1598] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:27:59.127988 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799ce94-576d-4e9f-85b9-8ab21d24f69d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:59.136030 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 793/1599] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:27:59.137164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10845d-1763-4ce8-a0e1-6a36ec06951d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:59.161680 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799ce94-576d-4e9f-85b9-8ab21d24f69d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:59.166788 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911c864-a5a0-4a5a-af65-07a35cebce05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:59.167210 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10845d-1763-4ce8-a0e1-6a36ec06951d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:59.167891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911c864-a5a0-4a5a-af65-07a35cebce05 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:59.168639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911c864-a5a0-4a5a-af65-07a35cebce05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:59.169184 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911c864-a5a0-4a5a-af65-07a35cebce05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:59.170634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9911c864-a5a0-4a5a-af65-07a35cebce05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:59.173859 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9911c864-a5a0-4a5a-af65-07a35cebce05 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:59.179301 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cfeaa-db62-40bb-9a1c-1c92bccd39f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:59.181681 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cfeaa-db62-40bb-9a1c-1c92bccd39f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:59.185144 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cfeaa-db62-40bb-9a1c-1c92bccd39f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:59.185686 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477cfeaa-db62-40bb-9a1c-1c92bccd39f7 admin admin] PATH_INFO: `/v3/projects/9c4e68e56e744fa9af4631ae2e967a6d/users/b7e13e8c76cb4757a42568112d16bd27/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:59.205497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 807/1600] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:27:59.206397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911c864-a5a0-4a5a-af65-07a35cebce05 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:59.207782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477cfeaa-db62-40bb-9a1c-1c92bccd39f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4e68e56e744fa9af4631ae2e967a6d, user_id=b7e13e8c76cb4757a42568112d16bd27, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:59.211055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477cfeaa-db62-40bb-9a1c-1c92bccd39f7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:59.218817 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911c864-a5a0-4a5a-af65-07a35cebce05 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:59.226272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927bb91-e78a-471c-90a3-c2b5ee8f0103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:59.227978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927bb91-e78a-471c-90a3-c2b5ee8f0103 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:59.228262 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927bb91-e78a-471c-90a3-c2b5ee8f0103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:59.228536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927bb91-e78a-471c-90a3-c2b5ee8f0103 admin admin] PATH_INFO: `/v3/projects/295dccb87b0c4f4a8dfd355b6b865894/users/29600664bdeb41349f95f1a39ded2096/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:59.236448 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 794/1601] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:27:59 2022] PUT /identity/v3/projects/9c4e68e56e744fa9af4631ae2e967a6d/users/b7e13e8c76cb4757a42568112d16bd27/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:27:59.243996 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cfeaa-db62-40bb-9a1c-1c92bccd39f7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:59.259747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927bb91-e78a-471c-90a3-c2b5ee8f0103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=295dccb87b0c4f4a8dfd355b6b865894, user_id=29600664bdeb41349f95f1a39ded2096, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:59.269501 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927bb91-e78a-471c-90a3-c2b5ee8f0103 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:59.290344 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 808/1602] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:27:59 2022] PUT /identity/v3/projects/295dccb87b0c4f4a8dfd355b6b865894/users/29600664bdeb41349f95f1a39ded2096/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:27:59.291190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927bb91-e78a-471c-90a3-c2b5ee8f0103 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:59.333858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:27:59.351242 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:27:59.352721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477cfeaa-db62-40bb-9a1c-1c92bccd39f7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:59.382471 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927bb91-e78a-471c-90a3-c2b5ee8f0103 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:59.388622 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50a6b7-ff0e-4dc2-94f6-edbe0d752894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:59.389305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50a6b7-ff0e-4dc2-94f6-edbe0d752894 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:59.389618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50a6b7-ff0e-4dc2-94f6-edbe0d752894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:59.389901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d50a6b7-ff0e-4dc2-94f6-edbe0d752894 admin admin] PATH_INFO: `/v3/projects/567bd0fe29da4db5a0f08c8b5cc81f9c/users/afbab63bf079403b8ef652422d581d25/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:59.423814 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50a6b7-ff0e-4dc2-94f6-edbe0d752894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=567bd0fe29da4db5a0f08c8b5cc81f9c, user_id=afbab63bf079403b8ef652422d581d25, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:59.428014 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d50a6b7-ff0e-4dc2-94f6-edbe0d752894 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:59.439420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ee003-b586-454d-9bc2-3e6851768fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:59.440181 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ee003-b586-454d-9bc2-3e6851768fe7 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:59.440515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ee003-b586-454d-9bc2-3e6851768fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:59.440893 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ee003-b586-454d-9bc2-3e6851768fe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:59.441849 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ee003-b586-454d-9bc2-3e6851768fe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:59.445043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98ee003-b586-454d-9bc2-3e6851768fe7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:59.452918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 809/1603] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:27:59 2022] PUT /identity/v3/projects/567bd0fe29da4db5a0f08c8b5cc81f9c/users/afbab63bf079403b8ef652422d581d25/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:27:59.454324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50a6b7-ff0e-4dc2-94f6-edbe0d752894 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:59.464413 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 795/1604] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:27:59 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:27:59.464413 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ee003-b586-454d-9bc2-3e6851768fe7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:59.477209 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ee003-b586-454d-9bc2-3e6851768fe7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:59.556497 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0770bc-25a4-4d4f-b76b-0e580716fad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:59.557361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0770bc-25a4-4d4f-b76b-0e580716fad8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:59.557686 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0770bc-25a4-4d4f-b76b-0e580716fad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:59.558169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0770bc-25a4-4d4f-b76b-0e580716fad8 admin admin] PATH_INFO: `/v3/projects/899fd4f1809241dcb0037945a74a0e62/users/8735a9bf71634488b4bfe9777d44dd5b/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:59.577076 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d50a6b7-ff0e-4dc2-94f6-edbe0d752894 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:59.580301 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0770bc-25a4-4d4f-b76b-0e580716fad8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=899fd4f1809241dcb0037945a74a0e62, user_id=8735a9bf71634488b4bfe9777d44dd5b, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:59.583521 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0770bc-25a4-4d4f-b76b-0e580716fad8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:59.584037 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3823d-381e-4435-8bf9-60b127515876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:59.585104 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3823d-381e-4435-8bf9-60b127515876 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:59.585650 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3823d-381e-4435-8bf9-60b127515876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:59.586242 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3823d-381e-4435-8bf9-60b127515876 admin admin] PATH_INFO: `/v3/projects/cb92681d2af8498cb7456a70354ccf39/users/4ed2359ca8cb4ee18984c14a5c3646b1/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:59.607544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 796/1605] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:27:59 2022] PUT /identity/v3/projects/899fd4f1809241dcb0037945a74a0e62/users/8735a9bf71634488b4bfe9777d44dd5b/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:27:59.610376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a5642-70ff-497a-b933-ee8ef6307d7c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:59.613947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a5642-70ff-497a-b933-ee8ef6307d7c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:59.614473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a5642-70ff-497a-b933-ee8ef6307d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:59.625591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3823d-381e-4435-8bf9-60b127515876 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb92681d2af8498cb7456a70354ccf39, user_id=4ed2359ca8cb4ee18984c14a5c3646b1, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:59.630596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3823d-381e-4435-8bf9-60b127515876 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:59.653816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 810/1606] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:27:59 2022] PUT /identity/v3/projects/cb92681d2af8498cb7456a70354ccf39/users/4ed2359ca8cb4ee18984c14a5c3646b1/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:27:59.655574 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e81abf-9a11-4686-bfcf-7b60eb890b00 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:59.656168 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e81abf-9a11-4686-bfcf-7b60eb890b00 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:59.656457 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e81abf-9a11-4686-bfcf-7b60eb890b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:59.671732 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c45a5642-70ff-497a-b933-ee8ef6307d7c None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:59.673027 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c45a5642-70ff-497a-b933-ee8ef6307d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:59.696972 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-68e81abf-9a11-4686-bfcf-7b60eb890b00 None None] MFA Rules not processed for user `29600664bdeb41349f95f1a39ded2096`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:59.698342 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-68e81abf-9a11-4686-bfcf-7b60eb890b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:59.804658 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 811/1607] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:59 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:27:59.811070 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b11e6-58f7-4991-b977-f77dd130b1d7 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:59.811275 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 797/1608] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:27:59.811367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b11e6-58f7-4991-b977-f77dd130b1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:59.811367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b11e6-58f7-4991-b977-f77dd130b1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:59.811532 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a5642-70ff-497a-b933-ee8ef6307d7c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:27:59.822054 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45a5642-70ff-497a-b933-ee8ef6307d7c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:27:59.827203 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc158d-a885-4c61-b1b1-94ea4eaf3824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:27:59.827886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc158d-a885-4c61-b1b1-94ea4eaf3824 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:59.828179 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc158d-a885-4c61-b1b1-94ea4eaf3824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:59.828439 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc158d-a885-4c61-b1b1-94ea4eaf3824 admin admin] PATH_INFO: `/v3/projects/ed2b4c6eb126468794c0aefdecb418ca/users/3eba3206e61043d3a81f49f71528bd7d/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:59.843563 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d36b11e6-58f7-4991-b977-f77dd130b1d7 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:59.846380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d36b11e6-58f7-4991-b977-f77dd130b1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:59.855804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:27:59.861748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc158d-a885-4c61-b1b1-94ea4eaf3824 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed2b4c6eb126468794c0aefdecb418ca, user_id=3eba3206e61043d3a81f49f71528bd7d, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:27:59.868797 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc158d-a885-4c61-b1b1-94ea4eaf3824 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:27:59.869728 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 812/1609] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:27:59.871196 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95f682-1334-4e5c-9e82-892dbf11f076 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:59.872079 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95f682-1334-4e5c-9e82-892dbf11f076 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:59.872927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95f682-1334-4e5c-9e82-892dbf11f076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:59.906751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 798/1610] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:27:59 2022] PUT /identity/v3/projects/ed2b4c6eb126468794c0aefdecb418ca/users/3eba3206e61043d3a81f49f71528bd7d/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:27:59.913657 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-fb95f682-1334-4e5c-9e82-892dbf11f076 None None] MFA Rules not processed for user `8735a9bf71634488b4bfe9777d44dd5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:59.915064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fb95f682-1334-4e5c-9e82-892dbf11f076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:27:59.915362 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154eabd0-0250-4bb3-bdcc-a636e9ae8069 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:27:59.916697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154eabd0-0250-4bb3-bdcc-a636e9ae8069 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:27:59.916962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154eabd0-0250-4bb3-bdcc-a636e9ae8069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:27:59.932262 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:27:59.952400 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-154eabd0-0250-4bb3-bdcc-a636e9ae8069 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:27:59.953637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-154eabd0-0250-4bb3-bdcc-a636e9ae8069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:00.048506 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 799/1611] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:00.049394 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154eabd0-0250-4bb3-bdcc-a636e9ae8069 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:00.064347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 813/1612] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:27:59 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:00.087167 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95f682-1334-4e5c-9e82-892dbf11f076 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:00.094757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154eabd0-0250-4bb3-bdcc-a636e9ae8069 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:00.107097 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423d457-b51b-42d8-b7c1-9ec68cc7604b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:00.110248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423d457-b51b-42d8-b7c1-9ec68cc7604b admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:00.110553 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423d457-b51b-42d8-b7c1-9ec68cc7604b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:00.110828 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423d457-b51b-42d8-b7c1-9ec68cc7604b admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:00.111524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5423d457-b51b-42d8-b7c1-9ec68cc7604b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:00.113954 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5423d457-b51b-42d8-b7c1-9ec68cc7604b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:00.163299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 800/1613] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:00 2022] POST /identity/v3/projects => generated 405 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:00.164052 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423d457-b51b-42d8-b7c1-9ec68cc7604b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:00.195555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95f682-1334-4e5c-9e82-892dbf11f076 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:00.287535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede809b5-abf0-40bb-9b05-4e13ac5183ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:00.288457 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede809b5-abf0-40bb-9b05-4e13ac5183ff service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:00.288750 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede809b5-abf0-40bb-9b05-4e13ac5183ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:00.289029 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede809b5-abf0-40bb-9b05-4e13ac5183ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:00.290747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423d457-b51b-42d8-b7c1-9ec68cc7604b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:00.296597 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3c3b2-a7c2-4647-8f05-1782e7245f27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:00.297429 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3c3b2-a7c2-4647-8f05-1782e7245f27 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:00.297803 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3c3b2-a7c2-4647-8f05-1782e7245f27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:00.298092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3c3b2-a7c2-4647-8f05-1782e7245f27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:00.367240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede809b5-abf0-40bb-9b05-4e13ac5183ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:00.368832 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede809b5-abf0-40bb-9b05-4e13ac5183ff service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:00.380774 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 814/1614] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:00.382561 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273fb0f-55ee-47ac-b324-a6f25cf948c3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:00.382964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273fb0f-55ee-47ac-b324-a6f25cf948c3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:00.383344 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273fb0f-55ee-47ac-b324-a6f25cf948c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:00.408586 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3c3b2-a7c2-4647-8f05-1782e7245f27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:00.410810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3c3b2-a7c2-4647-8f05-1782e7245f27 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:00.423585 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3273fb0f-55ee-47ac-b324-a6f25cf948c3 None None] MFA Rules not processed for user `3eba3206e61043d3a81f49f71528bd7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:00.424354 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 801/1615] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:00.425299 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3273fb0f-55ee-47ac-b324-a6f25cf948c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:00.425450 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3c3b2-a7c2-4647-8f05-1782e7245f27 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:00.436236 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3c3b2-a7c2-4647-8f05-1782e7245f27 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:00.436753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:00.442201 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b852f1-3fff-4cf5-a3fd-c9dcb9b66c16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:00.443226 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b852f1-3fff-4cf5-a3fd-c9dcb9b66c16 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:00.443756 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b852f1-3fff-4cf5-a3fd-c9dcb9b66c16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:00.444148 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b852f1-3fff-4cf5-a3fd-c9dcb9b66c16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:00.456149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b852f1-3fff-4cf5-a3fd-c9dcb9b66c16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:00.457622 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b852f1-3fff-4cf5-a3fd-c9dcb9b66c16 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:00.477799 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 802/1616] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:00.479247 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b852f1-3fff-4cf5-a3fd-c9dcb9b66c16 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:00.498510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 815/1617] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:00.499435 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273fb0f-55ee-47ac-b324-a6f25cf948c3 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:00.510994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3273fb0f-55ee-47ac-b324-a6f25cf948c3 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:00.511417 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:00.516899 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adabd3f-de53-431f-b152-5c167838abf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:00.517669 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adabd3f-de53-431f-b152-5c167838abf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:00.517961 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adabd3f-de53-431f-b152-5c167838abf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:00.518318 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adabd3f-de53-431f-b152-5c167838abf0 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:00.519822 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adabd3f-de53-431f-b152-5c167838abf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:00.522669 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adabd3f-de53-431f-b152-5c167838abf0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:00.554112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 816/1618] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:28:00 2022] POST /identity/v3/users => generated 309 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:00.559104 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20341f16-da7d-4a69-9cc3-14ea4b7ec960 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:00.559463 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20341f16-da7d-4a69-9cc3-14ea4b7ec960 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:00.559755 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20341f16-da7d-4a69-9cc3-14ea4b7ec960 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:00.561964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 817/1619] 10.222.0.39 () {54 vars in 962 bytes} [Thu Jul 28 22:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:28:00.562241 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20341f16-da7d-4a69-9cc3-14ea4b7ec960 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:00.580465 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20341f16-da7d-4a69-9cc3-14ea4b7ec960 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:00.586479 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93951fd4-5e91-4845-b286-b1a655f7bd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:00.587559 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93951fd4-5e91-4845-b286-b1a655f7bd92 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:00.588079 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93951fd4-5e91-4845-b286-b1a655f7bd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:00.588535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93951fd4-5e91-4845-b286-b1a655f7bd92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:00.589316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93951fd4-5e91-4845-b286-b1a655f7bd92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:00.592766 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93951fd4-5e91-4845-b286-b1a655f7bd92 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:00.612028 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 818/1620] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:00.616476 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be5ece-390d-4fdb-a987-f644c646ce55 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:00.616919 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be5ece-390d-4fdb-a987-f644c646ce55 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:00.617383 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be5ece-390d-4fdb-a987-f644c646ce55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:00.648998 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b852f1-3fff-4cf5-a3fd-c9dcb9b66c16 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:00.659411 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4d94f-a046-46cf-b0e1-6cb2dc5b2c7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:00.660105 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4d94f-a046-46cf-b0e1-6cb2dc5b2c7d service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:00.660450 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4d94f-a046-46cf-b0e1-6cb2dc5b2c7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:00.660725 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb4d94f-a046-46cf-b0e1-6cb2dc5b2c7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:00.746794 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-56be5ece-390d-4fdb-a987-f644c646ce55 None None] MFA Rules not processed for user `77efe8046ea0468cb0c7c58c24fc4e75`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:00.748080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-56be5ece-390d-4fdb-a987-f644c646ce55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:00.829576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4d94f-a046-46cf-b0e1-6cb2dc5b2c7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:00.831143 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb4d94f-a046-46cf-b0e1-6cb2dc5b2c7d service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:00.843317 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 803/1621] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:00 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:00.844221 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4d94f-a046-46cf-b0e1-6cb2dc5b2c7d service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:00.870745 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb4d94f-a046-46cf-b0e1-6cb2dc5b2c7d service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:00.882635 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d987f-43e9-4a12-9972-0ce65543d81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:00.885967 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d987f-43e9-4a12-9972-0ce65543d81a admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:00.886420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d987f-43e9-4a12-9972-0ce65543d81a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:00.886856 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189d987f-43e9-4a12-9972-0ce65543d81a admin admin] PATH_INFO: `/v3/projects/4484d46095914e1996a97193884687d4/users/e0a63425d9b44544b4004fb5c7a47e59/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:00.917262 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d987f-43e9-4a12-9972-0ce65543d81a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4484d46095914e1996a97193884687d4, user_id=e0a63425d9b44544b4004fb5c7a47e59, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:00.918643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 819/1622] 10.222.0.39 () {58 vars in 1059 bytes} [Thu Jul 28 22:28:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 306 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:00.932585 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189d987f-43e9-4a12-9972-0ce65543d81a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:00.942109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be5ece-390d-4fdb-a987-f644c646ce55 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:00.973207 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be5ece-390d-4fdb-a987-f644c646ce55 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:00.981328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894dceb-5e33-4b0e-b806-917d3af9326e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:00.982126 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894dceb-5e33-4b0e-b806-917d3af9326e service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:00.982450 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894dceb-5e33-4b0e-b806-917d3af9326e service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:00.983065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894dceb-5e33-4b0e-b806-917d3af9326e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:00.986948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 804/1623] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:00 2022] PUT /identity/v3/projects/4484d46095914e1996a97193884687d4/users/e0a63425d9b44544b4004fb5c7a47e59/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:00.991118 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d987f-43e9-4a12-9972-0ce65543d81a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:01.015355 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894dceb-5e33-4b0e-b806-917d3af9326e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:01.022267 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894dceb-5e33-4b0e-b806-917d3af9326e service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:01.042240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 820/1624] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:01.165202 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189d987f-43e9-4a12-9972-0ce65543d81a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:01.181381 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e8c6a-9c73-4983-8937-d697bd050858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:01.183454 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e8c6a-9c73-4983-8937-d697bd050858 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:01.183743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e8c6a-9c73-4983-8937-d697bd050858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:01.184312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e8c6a-9c73-4983-8937-d697bd050858 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:01.185023 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e8c6a-9c73-4983-8937-d697bd050858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:01.190142 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e8c6a-9c73-4983-8937-d697bd050858 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:01.216273 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 805/1625] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:00 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:01.217293 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894dceb-5e33-4b0e-b806-917d3af9326e service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:01.239707 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894dceb-5e33-4b0e-b806-917d3af9326e service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:01.252458 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0d8d8-21ba-4ee3-a83f-07e9c48ba1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:01.253156 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0d8d8-21ba-4ee3-a83f-07e9c48ba1e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:01.253450 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0d8d8-21ba-4ee3-a83f-07e9c48ba1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:01.256297 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a0d8d8-21ba-4ee3-a83f-07e9c48ba1e2 admin admin] PATH_INFO: `/v3/projects/4484d46095914e1996a97193884687d4/users/e0a63425d9b44544b4004fb5c7a47e59/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:01.260676 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a0d8d8-21ba-4ee3-a83f-07e9c48ba1e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4484d46095914e1996a97193884687d4, user_id=e0a63425d9b44544b4004fb5c7a47e59, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:01.266220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a0d8d8-21ba-4ee3-a83f-07e9c48ba1e2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:01.299502 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 821/1626] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:01 2022] PUT /identity/v3/projects/4484d46095914e1996a97193884687d4/users/e0a63425d9b44544b4004fb5c7a47e59/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:01.312165 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ab06b-641b-4808-991d-ab1e0ee4c15a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:01.312772 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ab06b-641b-4808-991d-ab1e0ee4c15a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:01.312772 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ab06b-641b-4808-991d-ab1e0ee4c15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:01.362895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3e4ab06b-641b-4808-991d-ab1e0ee4c15a None None] MFA Rules not processed for user `e0a63425d9b44544b4004fb5c7a47e59`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:01.364595 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3e4ab06b-641b-4808-991d-ab1e0ee4c15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:01.419741 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:01.585046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 806/1627] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:01 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:01.596240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0d8d8-21ba-4ee3-a83f-07e9c48ba1e2 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:01.696003 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:01.710006 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a5373-2561-4741-941e-3cd83f779155 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:01.710403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a5373-2561-4741-941e-3cd83f779155 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:01.710569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a5373-2561-4741-941e-3cd83f779155 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:01.719268 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 807/1628] 10.222.0.39 () {54 vars in 912 bytes} [Thu Jul 28 22:28:01 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:28:01.721089 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f2551-4847-487a-a42b-e9b654efae4d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:01.721689 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f2551-4847-487a-a42b-e9b654efae4d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:01.722137 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f2551-4847-487a-a42b-e9b654efae4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:01.762092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7a9f2551-4847-487a-a42b-e9b654efae4d None None] MFA Rules not processed for user `77efe8046ea0468cb0c7c58c24fc4e75`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:01.763343 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7a9f2551-4847-487a-a42b-e9b654efae4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:01.768191 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a0d8d8-21ba-4ee3-a83f-07e9c48ba1e2 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:01.781699 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cec7d-8042-49df-b404-4026d8581f20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:01.783951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cec7d-8042-49df-b404-4026d8581f20 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:01.784404 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cec7d-8042-49df-b404-4026d8581f20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:01.793272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4cec7d-8042-49df-b404-4026d8581f20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:01.838356 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4cec7d-8042-49df-b404-4026d8581f20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:01.841474 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4cec7d-8042-49df-b404-4026d8581f20 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:01.857349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 822/1629] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:01 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:01.945432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 808/1630] 10.222.0.39 () {58 vars in 1009 bytes} [Thu Jul 28 22:28:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:02.078725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cec7d-8042-49df-b404-4026d8581f20 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:02.099860 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4cec7d-8042-49df-b404-4026d8581f20 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:02.105382 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e966d-8216-4c44-8514-93c4031cf318 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:02.106407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e966d-8216-4c44-8514-93c4031cf318 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:02.106793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e966d-8216-4c44-8514-93c4031cf318 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:02.107166 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e966d-8216-4c44-8514-93c4031cf318 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:02.122094 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e966d-8216-4c44-8514-93c4031cf318 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:02.125117 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e966d-8216-4c44-8514-93c4031cf318 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:02.164311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 823/1631] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:02.444405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f2551-4847-487a-a42b-e9b654efae4d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:02.632700 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f2551-4847-487a-a42b-e9b654efae4d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:02.651500 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a60aae-504a-4ec1-952e-abd7df08e63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:02.665286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a60aae-504a-4ec1-952e-abd7df08e63b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:02.665692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a60aae-504a-4ec1-952e-abd7df08e63b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:02.666016 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a60aae-504a-4ec1-952e-abd7df08e63b admin admin] PATH_INFO: `/v3/users/29600664bdeb41349f95f1a39ded2096` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:02.674310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a60aae-504a-4ec1-952e-abd7df08e63b admin admin] RBAC: Authorizing `identity:delete_user(user_id=29600664bdeb41349f95f1a39ded2096)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:02.680680 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a60aae-504a-4ec1-952e-abd7df08e63b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:02.852207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-67a60aae-504a-4ec1-952e-abd7df08e63b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29600664bdeb41349f95f1a39ded2096'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:02.864172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:02.875871 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-67a60aae-504a-4ec1-952e-abd7df08e63b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29600664bdeb41349f95f1a39ded2096'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:02.877738 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-67a60aae-504a-4ec1-952e-abd7df08e63b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29600664bdeb41349f95f1a39ded2096'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:02.878732 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-67a60aae-504a-4ec1-952e-abd7df08e63b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29600664bdeb41349f95f1a39ded2096'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:02.956111 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-67a60aae-504a-4ec1-952e-abd7df08e63b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29600664bdeb41349f95f1a39ded2096'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:03.001373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 809/1632] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:28:02 2022] DELETE /identity/v3/users/29600664bdeb41349f95f1a39ded2096 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:03.007486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e90c9-317f-4e39-aad9-85f888a4a2c3 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:03.008924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e90c9-317f-4e39-aad9-85f888a4a2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:03.009474 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e90c9-317f-4e39-aad9-85f888a4a2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:03.057481 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-818e90c9-317f-4e39-aad9-85f888a4a2c3 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:03.058870 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-818e90c9-317f-4e39-aad9-85f888a4a2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:03.217672 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 824/1633] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:03.245141 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a60aae-504a-4ec1-952e-abd7df08e63b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:03.246624 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-67a60aae-504a-4ec1-952e-abd7df08e63b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:03.382407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601ff72-47f6-4990-ad5b-c7225bd6e73b None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:03.382787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601ff72-47f6-4990-ad5b-c7225bd6e73b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:03.383148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601ff72-47f6-4990-ad5b-c7225bd6e73b None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:03.385566 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 825/1634] 10.222.0.39 () {54 vars in 962 bytes} [Thu Jul 28 22:28:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:28:03.396234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a60aae-504a-4ec1-952e-abd7df08e63b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:03.400269 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7c8a2-13d6-4317-a758-4ddfc6e85e77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:03.400906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7c8a2-13d6-4317-a758-4ddfc6e85e77 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:03.401190 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7c8a2-13d6-4317-a758-4ddfc6e85e77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:03.401461 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7c8a2-13d6-4317-a758-4ddfc6e85e77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:03.412734 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532334b-bcc8-4914-948a-41d4bd79c8ed None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:03.413091 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532334b-bcc8-4914-948a-41d4bd79c8ed None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:03.413553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532334b-bcc8-4914-948a-41d4bd79c8ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:03.415274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7c8a2-13d6-4317-a758-4ddfc6e85e77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:03.417186 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7c8a2-13d6-4317-a758-4ddfc6e85e77 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:03.435539 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 810/1635] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:03.450057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8532334b-bcc8-4914-948a-41d4bd79c8ed None None] MFA Rules not processed for user `77efe8046ea0468cb0c7c58c24fc4e75`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:03.451295 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8532334b-bcc8-4914-948a-41d4bd79c8ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:03.630779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 826/1636] 10.222.0.39 () {58 vars in 1059 bytes} [Thu Jul 28 22:28:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:28:03.636670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f99127-7aae-4826-8ba3-16744e23ecb1 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:03.637648 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f99127-7aae-4826-8ba3-16744e23ecb1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:03.638134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f99127-7aae-4826-8ba3-16744e23ecb1 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:03.640133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 811/1637] 10.222.0.39 () {54 vars in 912 bytes} [Thu Jul 28 22:28:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:28:03.651483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d7f79-69e7-42bc-80e2-3b35223a7004 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:03.651785 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d7f79-69e7-42bc-80e2-3b35223a7004 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:03.653854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d7f79-69e7-42bc-80e2-3b35223a7004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:03.709263 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ad1d7f79-69e7-42bc-80e2-3b35223a7004 None None] MFA Rules not processed for user `77efe8046ea0468cb0c7c58c24fc4e75`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:03.711606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ad1d7f79-69e7-42bc-80e2-3b35223a7004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:03.729184 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 827/1638] 10.222.0.39 () {58 vars in 1009 bytes} [Thu Jul 28 22:28:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:03.749248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f99127-7aae-4826-8ba3-16744e23ecb1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:03.751798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b2f99127-7aae-4826-8ba3-16744e23ecb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:03.913805 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f99127-7aae-4826-8ba3-16744e23ecb1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:03.924574 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7ac4d-342e-44eb-ac80-d0e891a26133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:03.926058 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7ac4d-342e-44eb-ac80-d0e891a26133 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:03.926486 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7ac4d-342e-44eb-ac80-d0e891a26133 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:03.927459 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7ac4d-342e-44eb-ac80-d0e891a26133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:03.976773 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7ac4d-342e-44eb-ac80-d0e891a26133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:03.983628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7ac4d-342e-44eb-ac80-d0e891a26133 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:04.018749 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 812/1639] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:05.189940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d7f79-69e7-42bc-80e2-3b35223a7004 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:05.191900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ad1d7f79-69e7-42bc-80e2-3b35223a7004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:05.223841 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d7f79-69e7-42bc-80e2-3b35223a7004 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:05.234792 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566842a-04d5-4153-8642-b2491708847d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:05.235467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566842a-04d5-4153-8642-b2491708847d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:05.235750 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566842a-04d5-4153-8642-b2491708847d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:05.236029 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566842a-04d5-4153-8642-b2491708847d admin admin] PATH_INFO: `/v3/projects/295dccb87b0c4f4a8dfd355b6b865894` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:05.240110 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566842a-04d5-4153-8642-b2491708847d admin admin] RBAC: Authorizing `identity:delete_project(project_id=295dccb87b0c4f4a8dfd355b6b865894)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:05.242351 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0566842a-04d5-4153-8642-b2491708847d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:05.338929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:05.350592 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0566842a-04d5-4153-8642-b2491708847d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '295dccb87b0c4f4a8dfd355b6b865894', 'request_id': 'req-0566842a-04d5-4153-8642-b2491708847d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:05.367026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0566842a-04d5-4153-8642-b2491708847d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '295dccb87b0c4f4a8dfd355b6b865894', 'request_id': 'req-0566842a-04d5-4153-8642-b2491708847d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:05.373707 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0566842a-04d5-4153-8642-b2491708847d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '295dccb87b0c4f4a8dfd355b6b865894', 'request_id': 'req-0566842a-04d5-4153-8642-b2491708847d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:05.396341 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0566842a-04d5-4153-8642-b2491708847d admin admin] The token cache is being invalidate because project 295dccb87b0c4f4a8dfd355b6b865894 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:28:05.396761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0566842a-04d5-4153-8642-b2491708847d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:05.399717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0566842a-04d5-4153-8642-b2491708847d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:05.406636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 828/1640] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:28:05 2022] DELETE /identity/v3/projects/295dccb87b0c4f4a8dfd355b6b865894 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:05.411376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7ac4d-342e-44eb-ac80-d0e891a26133 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:05.430955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7ac4d-342e-44eb-ac80-d0e891a26133 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:05.592640 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbe6bef-3508-4863-99c8-3bb689181983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:05.593362 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe6bef-3508-4863-99c8-3bb689181983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:05.595543 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe6bef-3508-4863-99c8-3bb689181983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:05.595834 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe6bef-3508-4863-99c8-3bb689181983 admin admin] PATH_INFO: `/v3/users/e0a63425d9b44544b4004fb5c7a47e59` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:05.598121 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe6bef-3508-4863-99c8-3bb689181983 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0a63425d9b44544b4004fb5c7a47e59)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:05.600236 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbe6bef-3508-4863-99c8-3bb689181983 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:05.660377 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:05.719186 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fdbe6bef-3508-4863-99c8-3bb689181983 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0a63425d9b44544b4004fb5c7a47e59'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:05.736201 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fdbe6bef-3508-4863-99c8-3bb689181983 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0a63425d9b44544b4004fb5c7a47e59'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:05.737139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fdbe6bef-3508-4863-99c8-3bb689181983 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0a63425d9b44544b4004fb5c7a47e59'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:05.739578 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fdbe6bef-3508-4863-99c8-3bb689181983 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0a63425d9b44544b4004fb5c7a47e59'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:05.774920 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fdbe6bef-3508-4863-99c8-3bb689181983 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0a63425d9b44544b4004fb5c7a47e59'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:05.809895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 813/1641] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:28:05 2022] DELETE /identity/v3/users/e0a63425d9b44544b4004fb5c7a47e59 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:07.466166 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566842a-04d5-4153-8642-b2491708847d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:07.468434 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0566842a-04d5-4153-8642-b2491708847d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:07.527833 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566842a-04d5-4153-8642-b2491708847d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:07.703159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b0b70-380d-4fcf-864f-9c82a0d9c7b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:07.704504 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b0b70-380d-4fcf-864f-9c82a0d9c7b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:07.704827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b0b70-380d-4fcf-864f-9c82a0d9c7b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:07.705344 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b0b70-380d-4fcf-864f-9c82a0d9c7b8 admin admin] PATH_INFO: `/v3/projects/4484d46095914e1996a97193884687d4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:07.714030 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b0b70-380d-4fcf-864f-9c82a0d9c7b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4484d46095914e1996a97193884687d4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:07.720572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b0b70-380d-4fcf-864f-9c82a0d9c7b8 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:07.858004 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-415b0b70-380d-4fcf-864f-9c82a0d9c7b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4484d46095914e1996a97193884687d4', 'request_id': 'req-415b0b70-380d-4fcf-864f-9c82a0d9c7b8'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:07.880694 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-415b0b70-380d-4fcf-864f-9c82a0d9c7b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4484d46095914e1996a97193884687d4', 'request_id': 'req-415b0b70-380d-4fcf-864f-9c82a0d9c7b8'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:07.889300 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-415b0b70-380d-4fcf-864f-9c82a0d9c7b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4484d46095914e1996a97193884687d4', 'request_id': 'req-415b0b70-380d-4fcf-864f-9c82a0d9c7b8'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:07.907695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-415b0b70-380d-4fcf-864f-9c82a0d9c7b8 admin admin] The token cache is being invalidate because project 4484d46095914e1996a97193884687d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:28:07.911687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-415b0b70-380d-4fcf-864f-9c82a0d9c7b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:07.914706 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-415b0b70-380d-4fcf-864f-9c82a0d9c7b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:07.926316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 829/1642] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:28:07 2022] DELETE /identity/v3/projects/4484d46095914e1996a97193884687d4 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:07.929641 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8066545-466d-437b-8b2a-5022c7eab943 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:07.930072 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8066545-466d-437b-8b2a-5022c7eab943 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:07.930139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8066545-466d-437b-8b2a-5022c7eab943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:07.979002 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e8066545-466d-437b-8b2a-5022c7eab943 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:07.985249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e8066545-466d-437b-8b2a-5022c7eab943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:08.163898 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc9f06-b2e2-49a4-91e6-4e14e762d381 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:08.164623 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc9f06-b2e2-49a4-91e6-4e14e762d381 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:08.165058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc9f06-b2e2-49a4-91e6-4e14e762d381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:08.165365 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 814/1643] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:08.247300 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f0fc9f06-b2e2-49a4-91e6-4e14e762d381 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:08.253554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f0fc9f06-b2e2-49a4-91e6-4e14e762d381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:08.279874 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 830/1644] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:08.284688 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8066545-466d-437b-8b2a-5022c7eab943 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:08.305118 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8066545-466d-437b-8b2a-5022c7eab943 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:08.319083 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7fd6a-cff7-4668-b214-31988dbee901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:08.319751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7fd6a-cff7-4668-b214-31988dbee901 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:08.322339 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7fd6a-cff7-4668-b214-31988dbee901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:08.322696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b7fd6a-cff7-4668-b214-31988dbee901 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:08.323472 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7fd6a-cff7-4668-b214-31988dbee901 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:28:08.323788 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7fd6a-cff7-4668-b214-31988dbee901 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:08.340125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b7fd6a-cff7-4668-b214-31988dbee901 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:08.373900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc9f06-b2e2-49a4-91e6-4e14e762d381 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:08.395984 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 815/1645] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:28:08.438428 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc9f06-b2e2-49a4-91e6-4e14e762d381 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:08.460978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cec306-0c6a-4a9b-8f0b-04f5a03e9e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:08.461177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cec306-0c6a-4a9b-8f0b-04f5a03e9e44 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:08.461415 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cec306-0c6a-4a9b-8f0b-04f5a03e9e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:08.461674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cec306-0c6a-4a9b-8f0b-04f5a03e9e44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:08.462347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cec306-0c6a-4a9b-8f0b-04f5a03e9e44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:08.467509 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cec306-0c6a-4a9b-8f0b-04f5a03e9e44 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:08.555281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 831/1646] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:08 2022] POST /identity/v3/projects => generated 373 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:08.559671 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7fd6a-cff7-4668-b214-31988dbee901 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:08.580677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b7fd6a-cff7-4668-b214-31988dbee901 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:08.765334 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3bf07-0df8-44ff-98eb-7b0c2378bd65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:08.767885 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3bf07-0df8-44ff-98eb-7b0c2378bd65 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:08.769227 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3bf07-0df8-44ff-98eb-7b0c2378bd65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:08.771945 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa3bf07-0df8-44ff-98eb-7b0c2378bd65 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:08.773069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3bf07-0df8-44ff-98eb-7b0c2378bd65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:08.781271 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa3bf07-0df8-44ff-98eb-7b0c2378bd65 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:08.847986 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cec306-0c6a-4a9b-8f0b-04f5a03e9e44 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:08.856783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 816/1647] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:28:08 2022] POST /identity/v3/users => generated 287 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:08.865413 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cec306-0c6a-4a9b-8f0b-04f5a03e9e44 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:08.871170 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded01f6-2909-4f4a-ae9a-ee144fdd272a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:08.871980 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded01f6-2909-4f4a-ae9a-ee144fdd272a admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:08.872559 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded01f6-2909-4f4a-ae9a-ee144fdd272a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:08.873228 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded01f6-2909-4f4a-ae9a-ee144fdd272a admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:08.874154 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded01f6-2909-4f4a-ae9a-ee144fdd272a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:08.875926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aded01f6-2909-4f4a-ae9a-ee144fdd272a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:08.889726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 832/1648] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:28:08.894229 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3bf07-0df8-44ff-98eb-7b0c2378bd65 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:08.907577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa3bf07-0df8-44ff-98eb-7b0c2378bd65 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:08.916161 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a5a69-386a-4f96-98f3-05c5deeb50d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:08.918189 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a5a69-386a-4f96-98f3-05c5deeb50d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:08.918672 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a5a69-386a-4f96-98f3-05c5deeb50d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:08.919147 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a5a69-386a-4f96-98f3-05c5deeb50d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:08.920197 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a5a69-386a-4f96-98f3-05c5deeb50d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:08.922815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a5a69-386a-4f96-98f3-05c5deeb50d0 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:08.934116 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:08.949455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 817/1649] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:08 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:08.955883 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded01f6-2909-4f4a-ae9a-ee144fdd272a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:08.970296 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aded01f6-2909-4f4a-ae9a-ee144fdd272a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:08.977964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183e2e0-081e-488f-ba17-62c30d471ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:08.978909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183e2e0-081e-488f-ba17-62c30d471ba4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:08.979351 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183e2e0-081e-488f-ba17-62c30d471ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:08.979783 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183e2e0-081e-488f-ba17-62c30d471ba4 admin admin] PATH_INFO: `/v3/projects/9f811a23ceb241f2a489df2744a0bd7c/users/29ebc0e1eb2043c7a3b1c854df176a9e/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:09.009381 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d183e2e0-081e-488f-ba17-62c30d471ba4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f811a23ceb241f2a489df2744a0bd7c, user_id=29ebc0e1eb2043c7a3b1c854df176a9e, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:09.012518 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d183e2e0-081e-488f-ba17-62c30d471ba4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:09.031731 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:09.058213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 833/1650] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:08 2022] PUT /identity/v3/projects/9f811a23ceb241f2a489df2744a0bd7c/users/29ebc0e1eb2043c7a3b1c854df176a9e/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:09.067987 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db6293-424f-4bf6-83f0-135866020364 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:09.068282 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db6293-424f-4bf6-83f0-135866020364 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:09.068535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db6293-424f-4bf6-83f0-135866020364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:09.150911 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-37db6293-424f-4bf6-83f0-135866020364 None None] MFA Rules not processed for user `29ebc0e1eb2043c7a3b1c854df176a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:09.156380 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-37db6293-424f-4bf6-83f0-135866020364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:09.379140 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183e2e0-081e-488f-ba17-62c30d471ba4 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:09.381838 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 818/1651] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:28:09.610784 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183e2e0-081e-488f-ba17-62c30d471ba4 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:09.624975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c0689-ac55-4a85-9f9a-5a29be31ca12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:09.625670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c0689-ac55-4a85-9f9a-5a29be31ca12 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:09.625952 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c0689-ac55-4a85-9f9a-5a29be31ca12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:09.626225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c0689-ac55-4a85-9f9a-5a29be31ca12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:09.626918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c0689-ac55-4a85-9f9a-5a29be31ca12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:09.628879 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c0689-ac55-4a85-9f9a-5a29be31ca12 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:09.714876 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 834/1652] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:09 2022] POST /identity/v3/projects => generated 371 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:28:09.716127 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db6293-424f-4bf6-83f0-135866020364 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:09.737681 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db6293-424f-4bf6-83f0-135866020364 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:10.001126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c603a-c587-4e07-badd-a49550cc34cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:10.001842 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c603a-c587-4e07-badd-a49550cc34cf admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:10.002217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c603a-c587-4e07-badd-a49550cc34cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:10.002499 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819c603a-c587-4e07-badd-a49550cc34cf admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:10.003187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c603a-c587-4e07-badd-a49550cc34cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:10.006027 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819c603a-c587-4e07-badd-a49550cc34cf admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:10.136069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 819/1653] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:28:09 2022] POST /identity/v3/users => generated 292 bytes in 420 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:10.137365 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c0689-ac55-4a85-9f9a-5a29be31ca12 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:10.170967 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c0689-ac55-4a85-9f9a-5a29be31ca12 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:10.189198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6881ff-01cf-4f90-80eb-ae07b4c745e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:10.190576 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6881ff-01cf-4f90-80eb-ae07b4c745e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:10.191044 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6881ff-01cf-4f90-80eb-ae07b4c745e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:10.198193 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6881ff-01cf-4f90-80eb-ae07b4c745e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:10.199413 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6881ff-01cf-4f90-80eb-ae07b4c745e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:10.201659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6881ff-01cf-4f90-80eb-ae07b4c745e9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:10.253596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 835/1654] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:28:10.253911 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c603a-c587-4e07-badd-a49550cc34cf admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:10.277034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819c603a-c587-4e07-badd-a49550cc34cf admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:10.309730 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6728a-8db4-4973-aaa1-2037ec90a5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:10.310579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6728a-8db4-4973-aaa1-2037ec90a5d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:10.310868 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6728a-8db4-4973-aaa1-2037ec90a5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:10.311216 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f6728a-8db4-4973-aaa1-2037ec90a5d6 admin admin] PATH_INFO: `/v3/projects/c959e4167c3148fbaf05ed88b8d40af4/users/c5a2a826840b4345a18ed65845991d16/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:10.352852 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6728a-8db4-4973-aaa1-2037ec90a5d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c959e4167c3148fbaf05ed88b8d40af4, user_id=c5a2a826840b4345a18ed65845991d16, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:10.356638 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f6728a-8db4-4973-aaa1-2037ec90a5d6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:10.402064 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 820/1655] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:10 2022] PUT /identity/v3/projects/c959e4167c3148fbaf05ed88b8d40af4/users/c5a2a826840b4345a18ed65845991d16/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:28:10.405900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6881ff-01cf-4f90-80eb-ae07b4c745e9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:10.428562 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6881ff-01cf-4f90-80eb-ae07b4c745e9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:10.585515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb624ad1-032b-4366-b7e7-1cc0197639de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:10.587085 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb624ad1-032b-4366-b7e7-1cc0197639de admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:10.588819 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb624ad1-032b-4366-b7e7-1cc0197639de admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:10.589184 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb624ad1-032b-4366-b7e7-1cc0197639de admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:10.592002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb624ad1-032b-4366-b7e7-1cc0197639de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:10.596836 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb624ad1-032b-4366-b7e7-1cc0197639de admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:10.630157 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 836/1656] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:10.631970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6728a-8db4-4973-aaa1-2037ec90a5d6 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:10.654601 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f6728a-8db4-4973-aaa1-2037ec90a5d6 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:10.667398 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959048a3-ee79-467e-927a-8603e0a9cbd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:10.668101 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959048a3-ee79-467e-927a-8603e0a9cbd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:10.668386 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959048a3-ee79-467e-927a-8603e0a9cbd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:10.668663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959048a3-ee79-467e-927a-8603e0a9cbd9 admin admin] PATH_INFO: `/v3/projects/c959e4167c3148fbaf05ed88b8d40af4/users/c5a2a826840b4345a18ed65845991d16/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:10.674347 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959048a3-ee79-467e-927a-8603e0a9cbd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c959e4167c3148fbaf05ed88b8d40af4, user_id=c5a2a826840b4345a18ed65845991d16, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:10.677795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959048a3-ee79-467e-927a-8603e0a9cbd9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:10.713959 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 821/1657] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:10 2022] PUT /identity/v3/projects/c959e4167c3148fbaf05ed88b8d40af4/users/c5a2a826840b4345a18ed65845991d16/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:10.715978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080082e-6450-4c51-a9d7-cd90b9923316 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:10.716486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080082e-6450-4c51-a9d7-cd90b9923316 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:10.716852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b080082e-6450-4c51-a9d7-cd90b9923316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:10.814611 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b080082e-6450-4c51-a9d7-cd90b9923316 None None] MFA Rules not processed for user `c5a2a826840b4345a18ed65845991d16`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:10.820146 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b080082e-6450-4c51-a9d7-cd90b9923316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:11.007850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:11.029748 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 837/1658] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:10 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:11.039502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959048a3-ee79-467e-927a-8603e0a9cbd9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:11.041723 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-959048a3-ee79-467e-927a-8603e0a9cbd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:11.117443 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:11.142549 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959048a3-ee79-467e-927a-8603e0a9cbd9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:11.154553 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f31cd9-da69-409d-9f2e-644249ba9c0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:11.155738 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f31cd9-da69-409d-9f2e-644249ba9c0c service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:11.156284 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f31cd9-da69-409d-9f2e-644249ba9c0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:11.158114 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f31cd9-da69-409d-9f2e-644249ba9c0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:11.172748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f31cd9-da69-409d-9f2e-644249ba9c0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:11.174660 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f31cd9-da69-409d-9f2e-644249ba9c0c service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:11.190693 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 822/1659] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:11 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:12.256740 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fe452-8bf3-41d6-8785-3450f80fde41 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:12.257282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fe452-8bf3-41d6-8785-3450f80fde41 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:12.257926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9fe452-8bf3-41d6-8785-3450f80fde41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:12.327163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6f9fe452-8bf3-41d6-8785-3450f80fde41 None None] MFA Rules not processed for user `b7e13e8c76cb4757a42568112d16bd27`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:12.328316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6f9fe452-8bf3-41d6-8785-3450f80fde41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:12.361699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f31cd9-da69-409d-9f2e-644249ba9c0c service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:12.473576 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 838/1660] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:12.496224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fe452-8bf3-41d6-8785-3450f80fde41 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:12.499523 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6f9fe452-8bf3-41d6-8785-3450f80fde41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:12.530717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f31cd9-da69-409d-9f2e-644249ba9c0c service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:12.544537 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c92cc0-b16a-4802-a966-d478710a7232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:12.545229 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c92cc0-b16a-4802-a966-d478710a7232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:12.545516 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c92cc0-b16a-4802-a966-d478710a7232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:12.545816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c92cc0-b16a-4802-a966-d478710a7232 admin admin] PATH_INFO: `/v3/users/29ebc0e1eb2043c7a3b1c854df176a9e` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:12.549825 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c92cc0-b16a-4802-a966-d478710a7232 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29ebc0e1eb2043c7a3b1c854df176a9e)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:12.552497 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c92cc0-b16a-4802-a966-d478710a7232 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:12.671391 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d4c92cc0-b16a-4802-a966-d478710a7232 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29ebc0e1eb2043c7a3b1c854df176a9e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:12.699377 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d4c92cc0-b16a-4802-a966-d478710a7232 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29ebc0e1eb2043c7a3b1c854df176a9e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:12.701773 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d4c92cc0-b16a-4802-a966-d478710a7232 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29ebc0e1eb2043c7a3b1c854df176a9e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:12.702413 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d4c92cc0-b16a-4802-a966-d478710a7232 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29ebc0e1eb2043c7a3b1c854df176a9e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:12.721042 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9fe452-8bf3-41d6-8785-3450f80fde41 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:12.728498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d4c92cc0-b16a-4802-a966-d478710a7232 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29ebc0e1eb2043c7a3b1c854df176a9e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:12.739581 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592bba0-2627-4e94-8e5f-480a0efb812b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:12.740564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592bba0-2627-4e94-8e5f-480a0efb812b service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:12.740850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592bba0-2627-4e94-8e5f-480a0efb812b service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:12.741130 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592bba0-2627-4e94-8e5f-480a0efb812b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:12.745302 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9592bba0-2627-4e94-8e5f-480a0efb812b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:12.754302 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 823/1661] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:28:12 2022] DELETE /identity/v3/users/29ebc0e1eb2043c7a3b1c854df176a9e => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:12.760386 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e92d4-0ac5-4aa0-9b10-fc2ce205b282 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:12.760911 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e92d4-0ac5-4aa0-9b10-fc2ce205b282 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:12.761680 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2e92d4-0ac5-4aa0-9b10-fc2ce205b282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:12.771597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592bba0-2627-4e94-8e5f-480a0efb812b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:12.773570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592bba0-2627-4e94-8e5f-480a0efb812b service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:12.812920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 839/1662] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:12.844928 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0d2e92d4-0ac5-4aa0-9b10-fc2ce205b282 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:12.846169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0d2e92d4-0ac5-4aa0-9b10-fc2ce205b282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:12.927303 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 824/1663] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:12.942827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592bba0-2627-4e94-8e5f-480a0efb812b service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:12.946068 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9592bba0-2627-4e94-8e5f-480a0efb812b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:13.171228 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592bba0-2627-4e94-8e5f-480a0efb812b service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:13.198444 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e92d4-0ac5-4aa0-9b10-fc2ce205b282 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:13.198444 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0d2e92d4-0ac5-4aa0-9b10-fc2ce205b282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:13.210085 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:13.259989 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2e92d4-0ac5-4aa0-9b10-fc2ce205b282 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:13.268398 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:13.271396 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3375405d-ff3f-472d-b504-a34ea3b69c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:13.272146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375405d-ff3f-472d-b504-a34ea3b69c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:13.272429 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375405d-ff3f-472d-b504-a34ea3b69c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:13.276846 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375405d-ff3f-472d-b504-a34ea3b69c8b admin admin] PATH_INFO: `/v3/users/3eba3206e61043d3a81f49f71528bd7d` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:13.279759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375405d-ff3f-472d-b504-a34ea3b69c8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eba3206e61043d3a81f49f71528bd7d)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:13.292942 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375405d-ff3f-472d-b504-a34ea3b69c8b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:13.391170 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8216955e-2c49-44ca-bbd7-2c727d4e9321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:13.393728 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216955e-2c49-44ca-bbd7-2c727d4e9321 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:13.395034 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216955e-2c49-44ca-bbd7-2c727d4e9321 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:13.395719 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8216955e-2c49-44ca-bbd7-2c727d4e9321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:13.411953 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3375405d-ff3f-472d-b504-a34ea3b69c8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eba3206e61043d3a81f49f71528bd7d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:13.425605 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3375405d-ff3f-472d-b504-a34ea3b69c8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eba3206e61043d3a81f49f71528bd7d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:13.427720 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216955e-2c49-44ca-bbd7-2c727d4e9321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:13.428586 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3375405d-ff3f-472d-b504-a34ea3b69c8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eba3206e61043d3a81f49f71528bd7d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:13.432652 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8216955e-2c49-44ca-bbd7-2c727d4e9321 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:13.434017 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3375405d-ff3f-472d-b504-a34ea3b69c8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eba3206e61043d3a81f49f71528bd7d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:13.461456 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3375405d-ff3f-472d-b504-a34ea3b69c8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eba3206e61043d3a81f49f71528bd7d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:13.475478 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 840/1664] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:13.482471 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 825/1665] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:28:13 2022] DELETE /identity/v3/users/3eba3206e61043d3a81f49f71528bd7d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:13.487248 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d014173-b7ed-401c-a265-9f0cbec85331 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:13.487629 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d014173-b7ed-401c-a265-9f0cbec85331 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:13.487970 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d014173-b7ed-401c-a265-9f0cbec85331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:13.527272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3d014173-b7ed-401c-a265-9f0cbec85331 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:13.530486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3d014173-b7ed-401c-a265-9f0cbec85331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:13.692904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 841/1666] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:13.744216 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3375405d-ff3f-472d-b504-a34ea3b69c8b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:13.745943 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3375405d-ff3f-472d-b504-a34ea3b69c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:13.897776 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3375405d-ff3f-472d-b504-a34ea3b69c8b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:13.911353 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c52c617-d1ab-41f9-b666-631090f38f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:13.912263 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52c617-d1ab-41f9-b666-631090f38f87 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:13.912677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52c617-d1ab-41f9-b666-631090f38f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:13.913018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c52c617-d1ab-41f9-b666-631090f38f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:13.935103 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52c617-d1ab-41f9-b666-631090f38f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:13.936765 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c52c617-d1ab-41f9-b666-631090f38f87 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:13.975420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 826/1667] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:14.781421 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d014173-b7ed-401c-a265-9f0cbec85331 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:14.783551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3d014173-b7ed-401c-a265-9f0cbec85331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:14.814835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d014173-b7ed-401c-a265-9f0cbec85331 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:14.838230 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ec950-a5bb-4f26-9bd1-0375aea10df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:14.839287 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ec950-a5bb-4f26-9bd1-0375aea10df1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:14.844719 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ec950-a5bb-4f26-9bd1-0375aea10df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:14.845140 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300ec950-a5bb-4f26-9bd1-0375aea10df1 admin admin] PATH_INFO: `/v3/projects/9f811a23ceb241f2a489df2744a0bd7c` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:14.852204 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ec950-a5bb-4f26-9bd1-0375aea10df1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f811a23ceb241f2a489df2744a0bd7c)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:14.854714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300ec950-a5bb-4f26-9bd1-0375aea10df1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:14.889499 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16170f53-6364-446f-8c5e-bfb1ba07403b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:14.890120 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16170f53-6364-446f-8c5e-bfb1ba07403b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:14.892126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16170f53-6364-446f-8c5e-bfb1ba07403b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:14.961827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-16170f53-6364-446f-8c5e-bfb1ba07403b None None] MFA Rules not processed for user `7476eccb0a6f4195936bec75f7f80c7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:14.971886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-16170f53-6364-446f-8c5e-bfb1ba07403b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:15.012926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-300ec950-a5bb-4f26-9bd1-0375aea10df1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f811a23ceb241f2a489df2744a0bd7c', 'request_id': 'req-300ec950-a5bb-4f26-9bd1-0375aea10df1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:15.039821 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-300ec950-a5bb-4f26-9bd1-0375aea10df1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f811a23ceb241f2a489df2744a0bd7c', 'request_id': 'req-300ec950-a5bb-4f26-9bd1-0375aea10df1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:15.062263 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-300ec950-a5bb-4f26-9bd1-0375aea10df1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f811a23ceb241f2a489df2744a0bd7c', 'request_id': 'req-300ec950-a5bb-4f26-9bd1-0375aea10df1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:15.158869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-300ec950-a5bb-4f26-9bd1-0375aea10df1 admin admin] The token cache is being invalidate because project 9f811a23ceb241f2a489df2744a0bd7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:28:15.159177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-300ec950-a5bb-4f26-9bd1-0375aea10df1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:15.167896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-300ec950-a5bb-4f26-9bd1-0375aea10df1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:15.178598 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 842/1668] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:28:14 2022] DELETE /identity/v3/projects/9f811a23ceb241f2a489df2744a0bd7c => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:15.179838 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ec950-a5bb-4f26-9bd1-0375aea10df1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:15.184479 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-300ec950-a5bb-4f26-9bd1-0375aea10df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:15.200780 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 827/1669] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:28:15.209633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16170f53-6364-446f-8c5e-bfb1ba07403b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:15.211451 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-16170f53-6364-446f-8c5e-bfb1ba07403b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:15.268388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:15.330450 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:15.366493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300ec950-a5bb-4f26-9bd1-0375aea10df1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:15.377850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5620c-61c6-4010-b42d-86f8a62af21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:15.378554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5620c-61c6-4010-b42d-86f8a62af21d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:15.378845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5620c-61c6-4010-b42d-86f8a62af21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:15.379649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b5620c-61c6-4010-b42d-86f8a62af21d admin admin] PATH_INFO: `/v3/users/c5a2a826840b4345a18ed65845991d16` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:15.386438 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b5620c-61c6-4010-b42d-86f8a62af21d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5a2a826840b4345a18ed65845991d16)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:15.387501 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: WARNING py.warnings [None req-d1b5620c-61c6-4010-b42d-86f8a62af21d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 22:28:15.387501 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: warnings.warn(msg) Jul 28 22:28:15.387501 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: Jul 28 22:28:15.388740 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b5620c-61c6-4010-b42d-86f8a62af21d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:15.460954 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16170f53-6364-446f-8c5e-bfb1ba07403b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:15.478688 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f736ce0-c52f-46e0-93f0-374b2e19a631 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:15.479606 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f736ce0-c52f-46e0-93f0-374b2e19a631 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:15.480156 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f736ce0-c52f-46e0-93f0-374b2e19a631 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:15.481325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f736ce0-c52f-46e0-93f0-374b2e19a631 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:15.484409 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7f736ce0-c52f-46e0-93f0-374b2e19a631 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:15.502620 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f736ce0-c52f-46e0-93f0-374b2e19a631 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:15.508421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f736ce0-c52f-46e0-93f0-374b2e19a631 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:15.534565 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 828/1670] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:28:15.538460 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f736ce0-c52f-46e0-93f0-374b2e19a631 service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:15.549563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7f736ce0-c52f-46e0-93f0-374b2e19a631 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:15.565948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d1b5620c-61c6-4010-b42d-86f8a62af21d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5a2a826840b4345a18ed65845991d16'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:15.600773 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f736ce0-c52f-46e0-93f0-374b2e19a631 service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:15.601962 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d1b5620c-61c6-4010-b42d-86f8a62af21d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5a2a826840b4345a18ed65845991d16'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:15.623140 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4b0e42-d95f-4626-9967-81395b2aa73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:15.626985 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b0e42-d95f-4626-9967-81395b2aa73e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:15.630645 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b0e42-d95f-4626-9967-81395b2aa73e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:15.631100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4b0e42-d95f-4626-9967-81395b2aa73e admin admin] PATH_INFO: `/v3/projects/ed2b4c6eb126468794c0aefdecb418ca` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:15.633326 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b0e42-d95f-4626-9967-81395b2aa73e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed2b4c6eb126468794c0aefdecb418ca)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:15.636921 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-2c4b0e42-d95f-4626-9967-81395b2aa73e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 28 22:28:15.636921 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jul 28 22:28:15.636921 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: Jul 28 22:28:15.641117 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d1b5620c-61c6-4010-b42d-86f8a62af21d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5a2a826840b4345a18ed65845991d16'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:15.642960 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d1b5620c-61c6-4010-b42d-86f8a62af21d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5a2a826840b4345a18ed65845991d16'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:15.644431 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4b0e42-d95f-4626-9967-81395b2aa73e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:15.694119 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d1b5620c-61c6-4010-b42d-86f8a62af21d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5a2a826840b4345a18ed65845991d16'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:15.715390 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 843/1671] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:28:15 2022] DELETE /identity/v3/users/c5a2a826840b4345a18ed65845991d16 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:15.716419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5620c-61c6-4010-b42d-86f8a62af21d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:15.721574 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d1b5620c-61c6-4010-b42d-86f8a62af21d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:15.857061 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2c4b0e42-d95f-4626-9967-81395b2aa73e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed2b4c6eb126468794c0aefdecb418ca', 'request_id': 'req-2c4b0e42-d95f-4626-9967-81395b2aa73e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:15.879785 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2c4b0e42-d95f-4626-9967-81395b2aa73e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed2b4c6eb126468794c0aefdecb418ca', 'request_id': 'req-2c4b0e42-d95f-4626-9967-81395b2aa73e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:15.892047 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2c4b0e42-d95f-4626-9967-81395b2aa73e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed2b4c6eb126468794c0aefdecb418ca', 'request_id': 'req-2c4b0e42-d95f-4626-9967-81395b2aa73e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:15.921170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2c4b0e42-d95f-4626-9967-81395b2aa73e admin admin] The token cache is being invalidate because project ed2b4c6eb126468794c0aefdecb418ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:28:15.923324 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2c4b0e42-d95f-4626-9967-81395b2aa73e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:15.924574 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2c4b0e42-d95f-4626-9967-81395b2aa73e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:15.931624 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 829/1672] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:28:15 2022] DELETE /identity/v3/projects/ed2b4c6eb126468794c0aefdecb418ca => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:15.935700 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b5620c-61c6-4010-b42d-86f8a62af21d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:15.937427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184e468-ea38-41f0-9669-12423de6f75e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:15.937898 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184e468-ea38-41f0-9669-12423de6f75e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:15.938347 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184e468-ea38-41f0-9669-12423de6f75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:16.011869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c184e468-ea38-41f0-9669-12423de6f75e None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:16.013152 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c184e468-ea38-41f0-9669-12423de6f75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:16.158333 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa594f7-c04a-4158-bdc9-48565e22e2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:16.159044 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa594f7-c04a-4158-bdc9-48565e22e2f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:16.160340 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa594f7-c04a-4158-bdc9-48565e22e2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:16.161285 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa594f7-c04a-4158-bdc9-48565e22e2f6 admin admin] PATH_INFO: `/v3/users/8735a9bf71634488b4bfe9777d44dd5b` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:16.169350 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa594f7-c04a-4158-bdc9-48565e22e2f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8735a9bf71634488b4bfe9777d44dd5b)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:16.176781 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa594f7-c04a-4158-bdc9-48565e22e2f6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:16.224744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 830/1673] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:16.227390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c49f0-cace-4844-99da-ed6eab5656e0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:16.227690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c49f0-cace-4844-99da-ed6eab5656e0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:16.228053 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5c49f0-cace-4844-99da-ed6eab5656e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:16.257862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:16.288922 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4d5c49f0-cace-4844-99da-ed6eab5656e0 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:16.293407 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4d5c49f0-cace-4844-99da-ed6eab5656e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:16.320547 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 831/1674] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:16.321826 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c49f0-cace-4844-99da-ed6eab5656e0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:16.330776 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-aaa594f7-c04a-4158-bdc9-48565e22e2f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8735a9bf71634488b4bfe9777d44dd5b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:16.348697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5c49f0-cace-4844-99da-ed6eab5656e0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:16.356798 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-aaa594f7-c04a-4158-bdc9-48565e22e2f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8735a9bf71634488b4bfe9777d44dd5b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:16.377689 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-aaa594f7-c04a-4158-bdc9-48565e22e2f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8735a9bf71634488b4bfe9777d44dd5b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:16.380470 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-aaa594f7-c04a-4158-bdc9-48565e22e2f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8735a9bf71634488b4bfe9777d44dd5b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:16.385684 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320722d-5bc8-4002-b2ff-94fcf91ab8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:16.386815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320722d-5bc8-4002-b2ff-94fcf91ab8ba admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:16.387806 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320722d-5bc8-4002-b2ff-94fcf91ab8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:16.388113 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320722d-5bc8-4002-b2ff-94fcf91ab8ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:16.389012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320722d-5bc8-4002-b2ff-94fcf91ab8ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:28:16.389161 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320722d-5bc8-4002-b2ff-94fcf91ab8ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:16.399504 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320722d-5bc8-4002-b2ff-94fcf91ab8ba admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:16.440321 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:16.453292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-aaa594f7-c04a-4158-bdc9-48565e22e2f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8735a9bf71634488b4bfe9777d44dd5b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:16.465759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 844/1675] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:28:15 2022] DELETE /identity/v3/users/8735a9bf71634488b4bfe9777d44dd5b => generated 0 bytes in 750 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:16.466723 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 832/1676] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:28:16.470735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66637ef-5651-4b08-a1d0-6fef93382406 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:16.471031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66637ef-5651-4b08-a1d0-6fef93382406 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:16.471314 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66637ef-5651-4b08-a1d0-6fef93382406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:16.472916 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320722d-5bc8-4002-b2ff-94fcf91ab8ba admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:16.476881 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2320722d-5bc8-4002-b2ff-94fcf91ab8ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:16.500746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320722d-5bc8-4002-b2ff-94fcf91ab8ba admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:16.519191 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b66637ef-5651-4b08-a1d0-6fef93382406 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:16.521027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b66637ef-5651-4b08-a1d0-6fef93382406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:16.698936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbaea9a-a73f-49aa-ab7a-f5ff32d74e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:16.701019 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbaea9a-a73f-49aa-ab7a-f5ff32d74e51 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:16.707390 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 845/1677] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:16.707548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbaea9a-a73f-49aa-ab7a-f5ff32d74e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:16.707548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbaea9a-a73f-49aa-ab7a-f5ff32d74e51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:16.707548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbaea9a-a73f-49aa-ab7a-f5ff32d74e51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:16.713687 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbaea9a-a73f-49aa-ab7a-f5ff32d74e51 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:16.769135 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66637ef-5651-4b08-a1d0-6fef93382406 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:16.771038 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b66637ef-5651-4b08-a1d0-6fef93382406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:16.794391 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66637ef-5651-4b08-a1d0-6fef93382406 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:16.803285 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 833/1678] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:16 2022] POST /identity/v3/projects => generated 369 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:28:16.803549 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbaea9a-a73f-49aa-ab7a-f5ff32d74e51 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:16.808707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fcbaea9a-a73f-49aa-ab7a-f5ff32d74e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:16.809945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e704b83-79c2-4e10-b462-78ec27dbeb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:16.810633 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e704b83-79c2-4e10-b462-78ec27dbeb0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:16.810920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e704b83-79c2-4e10-b462-78ec27dbeb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:16.811197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e704b83-79c2-4e10-b462-78ec27dbeb0b admin admin] PATH_INFO: `/v3/projects/c959e4167c3148fbaf05ed88b8d40af4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:16.814825 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e704b83-79c2-4e10-b462-78ec27dbeb0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c959e4167c3148fbaf05ed88b8d40af4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:16.821845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e704b83-79c2-4e10-b462-78ec27dbeb0b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:16.964086 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbaea9a-a73f-49aa-ab7a-f5ff32d74e51 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:16.975600 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6e704b83-79c2-4e10-b462-78ec27dbeb0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c959e4167c3148fbaf05ed88b8d40af4', 'request_id': 'req-6e704b83-79c2-4e10-b462-78ec27dbeb0b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:17.000439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6e704b83-79c2-4e10-b462-78ec27dbeb0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c959e4167c3148fbaf05ed88b8d40af4', 'request_id': 'req-6e704b83-79c2-4e10-b462-78ec27dbeb0b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:17.021161 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6e704b83-79c2-4e10-b462-78ec27dbeb0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c959e4167c3148fbaf05ed88b8d40af4', 'request_id': 'req-6e704b83-79c2-4e10-b462-78ec27dbeb0b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:17.042425 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6e704b83-79c2-4e10-b462-78ec27dbeb0b admin admin] The token cache is being invalidate because project c959e4167c3148fbaf05ed88b8d40af4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:28:17.045283 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6e704b83-79c2-4e10-b462-78ec27dbeb0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:17.047480 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6e704b83-79c2-4e10-b462-78ec27dbeb0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:17.065186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 846/1679] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:28:16 2022] DELETE /identity/v3/projects/c959e4167c3148fbaf05ed88b8d40af4 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:28:17.066349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e704b83-79c2-4e10-b462-78ec27dbeb0b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:17.071155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6e704b83-79c2-4e10-b462-78ec27dbeb0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:17.097312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d752567-b0b4-4af9-bd12-35f1461e1dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:17.099812 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d752567-b0b4-4af9-bd12-35f1461e1dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:17.101504 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d752567-b0b4-4af9-bd12-35f1461e1dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:17.102143 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d752567-b0b4-4af9-bd12-35f1461e1dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:17.106488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3d752567-b0b4-4af9-bd12-35f1461e1dd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:17.165059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:17.281341 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:17.296186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e704b83-79c2-4e10-b462-78ec27dbeb0b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:17.313137 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506853ca-8be1-4922-b95a-66454e8cc78c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:17.314975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506853ca-8be1-4922-b95a-66454e8cc78c admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:17.315258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506853ca-8be1-4922-b95a-66454e8cc78c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:17.315637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d752567-b0b4-4af9-bd12-35f1461e1dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:17.315959 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506853ca-8be1-4922-b95a-66454e8cc78c admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:17.318482 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d752567-b0b4-4af9-bd12-35f1461e1dd3 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:17.318926 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506853ca-8be1-4922-b95a-66454e8cc78c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:17.322465 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506853ca-8be1-4922-b95a-66454e8cc78c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:17.346146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 834/1680] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:17.350588 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548416bb-483d-43ac-905e-5d0c53131b62 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:17.351146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548416bb-483d-43ac-905e-5d0c53131b62 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:17.352600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548416bb-483d-43ac-905e-5d0c53131b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:17.377810 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 847/1681] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:28:17 2022] POST /identity/v3/users => generated 285 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:17.383147 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506853ca-8be1-4922-b95a-66454e8cc78c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:17.414452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506853ca-8be1-4922-b95a-66454e8cc78c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:17.415688 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-548416bb-483d-43ac-905e-5d0c53131b62 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:17.417969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-548416bb-483d-43ac-905e-5d0c53131b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:17.427591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2cc7f-85e0-4302-9682-eea9dfdee10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:17.428430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2cc7f-85e0-4302-9682-eea9dfdee10b admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:17.428791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2cc7f-85e0-4302-9682-eea9dfdee10b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:17.428982 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d2cc7f-85e0-4302-9682-eea9dfdee10b admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:17.429726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2cc7f-85e0-4302-9682-eea9dfdee10b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:17.431390 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d2cc7f-85e0-4302-9682-eea9dfdee10b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:17.451365 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 835/1682] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:17.453915 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a1928-1cfc-42f4-b730-0fd4f9a71833 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:17.454215 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a1928-1cfc-42f4-b730-0fd4f9a71833 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:17.454676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5a1928-1cfc-42f4-b730-0fd4f9a71833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:17.470434 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 848/1683] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:17.475047 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2cc7f-85e0-4302-9682-eea9dfdee10b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:17.495815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3d5a1928-1cfc-42f4-b730-0fd4f9a71833 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:17.497386 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3d5a1928-1cfc-42f4-b730-0fd4f9a71833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:17.499724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d2cc7f-85e0-4302-9682-eea9dfdee10b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:17.506174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45141d-55db-4fb5-af6e-8538a3874c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:17.506821 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45141d-55db-4fb5-af6e-8538a3874c9b admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:17.507107 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45141d-55db-4fb5-af6e-8538a3874c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:17.507389 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45141d-55db-4fb5-af6e-8538a3874c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:17.508251 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45141d-55db-4fb5-af6e-8538a3874c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:17.512024 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 836/1684] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:17.512161 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45141d-55db-4fb5-af6e-8538a3874c9b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:17.515079 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a1928-1cfc-42f4-b730-0fd4f9a71833 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:17.536704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5a1928-1cfc-42f4-b730-0fd4f9a71833 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:17.537810 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 849/1685] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:28:17.542510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45141d-55db-4fb5-af6e-8538a3874c9b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:17.547936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53000666-9cf5-4a06-9f26-97211f5c4a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:17.548582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53000666-9cf5-4a06-9f26-97211f5c4a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:17.549031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53000666-9cf5-4a06-9f26-97211f5c4a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:17.549319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53000666-9cf5-4a06-9f26-97211f5c4a1a admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:17.550104 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53000666-9cf5-4a06-9f26-97211f5c4a1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:28:17.552523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53000666-9cf5-4a06-9f26-97211f5c4a1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:17.556882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53000666-9cf5-4a06-9f26-97211f5c4a1a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:17.563012 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45141d-55db-4fb5-af6e-8538a3874c9b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:17.578815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 837/1686] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:28:17.579104 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2951282c-4e1b-4da4-86fd-4dc996560b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:17.579800 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951282c-4e1b-4da4-86fd-4dc996560b01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:17.580084 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951282c-4e1b-4da4-86fd-4dc996560b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:17.580202 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53000666-9cf5-4a06-9f26-97211f5c4a1a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:17.580356 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2951282c-4e1b-4da4-86fd-4dc996560b01 admin admin] PATH_INFO: `/v3/projects/bc392ca648a74cd1a9dc1171709083ab/users/72eefaed2c7f4151a01bfe4d80efe92f/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:17.594936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53000666-9cf5-4a06-9f26-97211f5c4a1a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:17.602149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655fed6-10c2-4b17-93de-33e5573334e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:17.602860 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655fed6-10c2-4b17-93de-33e5573334e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:17.603260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655fed6-10c2-4b17-93de-33e5573334e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:17.603617 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655fed6-10c2-4b17-93de-33e5573334e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:17.605607 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655fed6-10c2-4b17-93de-33e5573334e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:17.608330 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655fed6-10c2-4b17-93de-33e5573334e8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:17.614843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951282c-4e1b-4da4-86fd-4dc996560b01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc392ca648a74cd1a9dc1171709083ab, user_id=72eefaed2c7f4151a01bfe4d80efe92f, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:17.618088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2951282c-4e1b-4da4-86fd-4dc996560b01 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:17.645653 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 850/1687] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:17 2022] PUT /identity/v3/projects/bc392ca648a74cd1a9dc1171709083ab/users/72eefaed2c7f4151a01bfe4d80efe92f/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:17.660093 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcebef5-2968-4382-9bb2-18a05372aeb1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:17.660510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcebef5-2968-4382-9bb2-18a05372aeb1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:17.661415 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcebef5-2968-4382-9bb2-18a05372aeb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:17.681912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 838/1688] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:17 2022] POST /identity/v3/projects => generated 355 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:17.683420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655fed6-10c2-4b17-93de-33e5573334e8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:17.734937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-edcebef5-2968-4382-9bb2-18a05372aeb1 None None] MFA Rules not processed for user `72eefaed2c7f4151a01bfe4d80efe92f`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:17.736302 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-edcebef5-2968-4382-9bb2-18a05372aeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:17.874989 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655fed6-10c2-4b17-93de-33e5573334e8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:17.883112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 851/1689] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:17 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:28:17.885074 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcebef5-2968-4382-9bb2-18a05372aeb1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:17.889066 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9680c-fcf4-4e41-95ba-e1482069fdcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:17.889866 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9680c-fcf4-4e41-95ba-e1482069fdcd admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:17.890253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9680c-fcf4-4e41-95ba-e1482069fdcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:17.890636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9680c-fcf4-4e41-95ba-e1482069fdcd admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:17.891418 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9680c-fcf4-4e41-95ba-e1482069fdcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:17.896710 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9680c-fcf4-4e41-95ba-e1482069fdcd admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:17.908582 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcebef5-2968-4382-9bb2-18a05372aeb1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:17.917664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27ee77-492d-48fd-ac01-dc55bd3d2ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:17.918230 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27ee77-492d-48fd-ac01-dc55bd3d2ffb admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:17.918648 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27ee77-492d-48fd-ac01-dc55bd3d2ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:17.918931 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec27ee77-492d-48fd-ac01-dc55bd3d2ffb admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:17.919620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27ee77-492d-48fd-ac01-dc55bd3d2ffb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:17.924288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec27ee77-492d-48fd-ac01-dc55bd3d2ffb admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:17.951472 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 839/1690] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:28:17 2022] POST /identity/v3/users => generated 278 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:17.955882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9680c-fcf4-4e41-95ba-e1482069fdcd admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:17.981642 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9680c-fcf4-4e41-95ba-e1482069fdcd admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:17.992737 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 852/1691] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:17 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:17.995415 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27ee77-492d-48fd-ac01-dc55bd3d2ffb admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:17.998956 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9e681-a940-463e-8a6a-adc2902c5818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:18.000362 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9e681-a940-463e-8a6a-adc2902c5818 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:18.000362 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9e681-a940-463e-8a6a-adc2902c5818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:18.000362 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9e681-a940-463e-8a6a-adc2902c5818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:18.001616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9e681-a940-463e-8a6a-adc2902c5818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:18.007333 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9e681-a940-463e-8a6a-adc2902c5818 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:18.041252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 840/1692] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:18.042872 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9e681-a940-463e-8a6a-adc2902c5818 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:18.068305 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9e681-a940-463e-8a6a-adc2902c5818 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:18.085888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:18.175002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec27ee77-492d-48fd-ac01-dc55bd3d2ffb admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:18.190369 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d59b5e-7a86-45e0-a7c4-83d71ad79d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:18.190881 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d59b5e-7a86-45e0-a7c4-83d71ad79d30 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:18.193437 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d59b5e-7a86-45e0-a7c4-83d71ad79d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:18.193807 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d59b5e-7a86-45e0-a7c4-83d71ad79d30 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:18.195669 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d59b5e-7a86-45e0-a7c4-83d71ad79d30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:18.201331 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d59b5e-7a86-45e0-a7c4-83d71ad79d30 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:18.245487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfed3e-048a-47bc-99b5-4d41af886a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:18.246346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfed3e-048a-47bc-99b5-4d41af886a7e admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:18.253832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfed3e-048a-47bc-99b5-4d41af886a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:18.254990 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfed3e-048a-47bc-99b5-4d41af886a7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:18.257059 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dfed3e-048a-47bc-99b5-4d41af886a7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:18.262232 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dfed3e-048a-47bc-99b5-4d41af886a7e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:18.264845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 853/1693] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:28:17 2022] POST /identity/v3/users => generated 290 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:28:18.268808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d59b5e-7a86-45e0-a7c4-83d71ad79d30 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:18.284678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 841/1694] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:18.290409 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfed3e-048a-47bc-99b5-4d41af886a7e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:18.296938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d59b5e-7a86-45e0-a7c4-83d71ad79d30 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:18.314530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfed3e-048a-47bc-99b5-4d41af886a7e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:18.333827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd764f-4b39-49b0-b838-9dd0763355fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:18.334847 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149cfcf6-988e-4e9d-9c24-609f57772362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:18.335651 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cfcf6-988e-4e9d-9c24-609f57772362 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:18.336043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cfcf6-988e-4e9d-9c24-609f57772362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:18.337131 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cfcf6-988e-4e9d-9c24-609f57772362 admin admin] PATH_INFO: `/v3/projects/bfa5bfff9d9044198a05f52d08356011/users/d52f5056cf0e49c2945fd718f19af77e/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:18.337692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd764f-4b39-49b0-b838-9dd0763355fe admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:18.338004 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd764f-4b39-49b0-b838-9dd0763355fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:18.338291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fd764f-4b39-49b0-b838-9dd0763355fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:18.340889 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fd764f-4b39-49b0-b838-9dd0763355fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:18.344990 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fd764f-4b39-49b0-b838-9dd0763355fe admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:18.375933 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:18.383257 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 854/1695] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:18.387462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149cfcf6-988e-4e9d-9c24-609f57772362 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa5bfff9d9044198a05f52d08356011, user_id=d52f5056cf0e49c2945fd718f19af77e, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:18.387652 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd764f-4b39-49b0-b838-9dd0763355fe admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:18.391473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149cfcf6-988e-4e9d-9c24-609f57772362 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:18.419721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 842/1696] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:18 2022] PUT /identity/v3/projects/bfa5bfff9d9044198a05f52d08356011/users/d52f5056cf0e49c2945fd718f19af77e/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:18.423629 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fd764f-4b39-49b0-b838-9dd0763355fe admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:18.432198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d7366-8509-4e5f-9907-d0ce6c3846db None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:18.432775 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d7366-8509-4e5f-9907-d0ce6c3846db None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:18.433164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d7366-8509-4e5f-9907-d0ce6c3846db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:18.527990 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-000d7366-8509-4e5f-9907-d0ce6c3846db None None] MFA Rules not processed for user `d52f5056cf0e49c2945fd718f19af77e`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:18.529423 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-000d7366-8509-4e5f-9907-d0ce6c3846db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:18.623620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604e35c-c4b9-418d-ac02-1bd32918b73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:18.624316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604e35c-c4b9-418d-ac02-1bd32918b73d admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:18.624604 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604e35c-c4b9-418d-ac02-1bd32918b73d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:18.624890 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b604e35c-c4b9-418d-ac02-1bd32918b73d admin admin] PATH_INFO: `/v3/projects/c5a06c7e3a244c9ab99f404d69a9dae9/users/f693151d830d4826b4694a11afd87340/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:18.667577 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b604e35c-c4b9-418d-ac02-1bd32918b73d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5a06c7e3a244c9ab99f404d69a9dae9, user_id=f693151d830d4826b4694a11afd87340, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:18.670610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b604e35c-c4b9-418d-ac02-1bd32918b73d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:18.709158 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 843/1697] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:18 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:18.711584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d7366-8509-4e5f-9907-d0ce6c3846db None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:18.712030 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 855/1698] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:18 2022] PUT /identity/v3/projects/c5a06c7e3a244c9ab99f404d69a9dae9/users/f693151d830d4826b4694a11afd87340/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:18.716714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604e35c-c4b9-418d-ac02-1bd32918b73d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:18.746227 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d7366-8509-4e5f-9907-d0ce6c3846db None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:18.909403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c77dcf-4c9d-4f4a-86e7-4f8efe9118ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:18.910205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c77dcf-4c9d-4f4a-86e7-4f8efe9118ef admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:18.911548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c77dcf-4c9d-4f4a-86e7-4f8efe9118ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:18.911959 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c77dcf-4c9d-4f4a-86e7-4f8efe9118ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:18.912772 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c77dcf-4c9d-4f4a-86e7-4f8efe9118ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:18.918728 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c77dcf-4c9d-4f4a-86e7-4f8efe9118ef admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:18.921047 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b604e35c-c4b9-418d-ac02-1bd32918b73d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:18.935497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482ee74-5615-4aa5-bab1-c1b834ee8c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:18.936155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482ee74-5615-4aa5-bab1-c1b834ee8c94 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:18.936440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482ee74-5615-4aa5-bab1-c1b834ee8c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:18.936728 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7482ee74-5615-4aa5-bab1-c1b834ee8c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:18.937433 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482ee74-5615-4aa5-bab1-c1b834ee8c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:18.939107 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7482ee74-5615-4aa5-bab1-c1b834ee8c94 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:18.976497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 856/1699] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:18.978577 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482ee74-5615-4aa5-bab1-c1b834ee8c94 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:19.006428 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7482ee74-5615-4aa5-bab1-c1b834ee8c94 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:19.007090 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 844/1700] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:18 2022] POST /identity/v3/projects => generated 357 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:19.007512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c77dcf-4c9d-4f4a-86e7-4f8efe9118ef admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:19.084154 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:19.168342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c77dcf-4c9d-4f4a-86e7-4f8efe9118ef admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:19.181139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c77af4-20bf-43b2-883a-7615196fc904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:19.181849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c77af4-20bf-43b2-883a-7615196fc904 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:19.182136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c77af4-20bf-43b2-883a-7615196fc904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:19.182420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c77af4-20bf-43b2-883a-7615196fc904 admin admin] PATH_INFO: `/v3/projects/c5a06c7e3a244c9ab99f404d69a9dae9/users/f693151d830d4826b4694a11afd87340/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:19.185097 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c412f8-8279-4ff5-ac8c-f143e870b64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:19.185767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c412f8-8279-4ff5-ac8c-f143e870b64a admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:19.186169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c412f8-8279-4ff5-ac8c-f143e870b64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:19.186380 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c412f8-8279-4ff5-ac8c-f143e870b64a admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:19.187061 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c412f8-8279-4ff5-ac8c-f143e870b64a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:19.189041 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c412f8-8279-4ff5-ac8c-f143e870b64a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:19.194192 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c77af4-20bf-43b2-883a-7615196fc904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5a06c7e3a244c9ab99f404d69a9dae9, user_id=f693151d830d4826b4694a11afd87340, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:19.197280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c77af4-20bf-43b2-883a-7615196fc904 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:19.218503 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 857/1701] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:18 2022] PUT /identity/v3/projects/c5a06c7e3a244c9ab99f404d69a9dae9/users/f693151d830d4826b4694a11afd87340/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:19.227381 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76389cb3-d33f-4fe2-917c-e3a26de21f7c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:19.227381 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76389cb3-d33f-4fe2-917c-e3a26de21f7c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:19.227381 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76389cb3-d33f-4fe2-917c-e3a26de21f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:19.229616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:19.265953 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 845/1702] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:28:19 2022] POST /identity/v3/users => generated 285 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:19.267615 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58504f-e601-463f-9a72-b5f0b8281803 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:19.273903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58504f-e601-463f-9a72-b5f0b8281803 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:19.274219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58504f-e601-463f-9a72-b5f0b8281803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:19.276187 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-76389cb3-d33f-4fe2-917c-e3a26de21f7c None None] MFA Rules not processed for user `b7e13e8c76cb4757a42568112d16bd27`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:19.278083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-76389cb3-d33f-4fe2-917c-e3a26de21f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:19.350890 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-bb58504f-e601-463f-9a72-b5f0b8281803 None None] MFA Rules not processed for user `f693151d830d4826b4694a11afd87340`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:19.352161 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bb58504f-e601-463f-9a72-b5f0b8281803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:19.425808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 858/1703] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:19.433038 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76389cb3-d33f-4fe2-917c-e3a26de21f7c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:19.518929 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 846/1704] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:19 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:19.521666 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fee98a-ddc8-4402-aaeb-cd7452908d0d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:19.522239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fee98a-ddc8-4402-aaeb-cd7452908d0d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:19.522752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fee98a-ddc8-4402-aaeb-cd7452908d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:19.569635 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-45fee98a-ddc8-4402-aaeb-cd7452908d0d None None] MFA Rules not processed for user `d38b517910894f7fa4351a2120b466d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:19.571154 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-45fee98a-ddc8-4402-aaeb-cd7452908d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:19.590974 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76389cb3-d33f-4fe2-917c-e3a26de21f7c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:19.603954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6407dbd-a3f8-4f8b-8b9f-04368561b619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:19.604602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6407dbd-a3f8-4f8b-8b9f-04368561b619 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:19.604885 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6407dbd-a3f8-4f8b-8b9f-04368561b619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:19.605161 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6407dbd-a3f8-4f8b-8b9f-04368561b619 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:19.606232 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6407dbd-a3f8-4f8b-8b9f-04368561b619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:19.612192 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6407dbd-a3f8-4f8b-8b9f-04368561b619 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:19.652153 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 859/1705] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:19.653064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6407dbd-a3f8-4f8b-8b9f-04368561b619 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:19.663436 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c6407dbd-a3f8-4f8b-8b9f-04368561b619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:19.742675 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 847/1706] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:19.753180 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fee98a-ddc8-4402-aaeb-cd7452908d0d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:19.790107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fee98a-ddc8-4402-aaeb-cd7452908d0d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:19.808400 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15440e3-56b8-4606-9f4d-f0bb6bb25fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:19.809183 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15440e3-56b8-4606-9f4d-f0bb6bb25fbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:19.810116 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15440e3-56b8-4606-9f4d-f0bb6bb25fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:19.811716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15440e3-56b8-4606-9f4d-f0bb6bb25fbe admin admin] PATH_INFO: `/v3/projects/b3aaa3e46bba42d9a66c1ba8d355d4ec/users/5b4c41a94ca64d2c80f4a7522b8ff2ac/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:19.882296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15440e3-56b8-4606-9f4d-f0bb6bb25fbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3aaa3e46bba42d9a66c1ba8d355d4ec, user_id=5b4c41a94ca64d2c80f4a7522b8ff2ac, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:19.888591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15440e3-56b8-4606-9f4d-f0bb6bb25fbe admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:19.900106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6407dbd-a3f8-4f8b-8b9f-04368561b619 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:19.912396 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad7c35-34e5-41b7-8b78-96020880b848 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:19.913101 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad7c35-34e5-41b7-8b78-96020880b848 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:19.913394 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad7c35-34e5-41b7-8b78-96020880b848 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:19.913713 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad7c35-34e5-41b7-8b78-96020880b848 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:19.931914 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 848/1707] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:19 2022] PUT /identity/v3/projects/b3aaa3e46bba42d9a66c1ba8d355d4ec/users/5b4c41a94ca64d2c80f4a7522b8ff2ac/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:19.931914 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15440e3-56b8-4606-9f4d-f0bb6bb25fbe admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:19.938022 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b15440e3-56b8-4606-9f4d-f0bb6bb25fbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:19.959606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad7c35-34e5-41b7-8b78-96020880b848 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:19.961889 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad7c35-34e5-41b7-8b78-96020880b848 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:20.010342 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 860/1708] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:19 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:20.011359 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad7c35-34e5-41b7-8b78-96020880b848 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:20.069879 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad7c35-34e5-41b7-8b78-96020880b848 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:20.094133 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:20.190996 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:20.248108 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15440e3-56b8-4606-9f4d-f0bb6bb25fbe admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:20.255777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34e4df1-1391-412a-930d-71882a1340a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:20.256588 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e4df1-1391-412a-930d-71882a1340a6 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:20.256954 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e4df1-1391-412a-930d-71882a1340a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:20.257324 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34e4df1-1391-412a-930d-71882a1340a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:20.313228 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd68ec-1c78-43e6-96b9-f483276ca91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:20.314013 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd68ec-1c78-43e6-96b9-f483276ca91b admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:20.314355 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd68ec-1c78-43e6-96b9-f483276ca91b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:20.314704 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbd68ec-1c78-43e6-96b9-f483276ca91b admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:20.317419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd68ec-1c78-43e6-96b9-f483276ca91b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:20.319656 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbd68ec-1c78-43e6-96b9-f483276ca91b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:20.374313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 861/1709] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:20.375302 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd68ec-1c78-43e6-96b9-f483276ca91b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:20.377074 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ffbd68ec-1c78-43e6-96b9-f483276ca91b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:20.399955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbd68ec-1c78-43e6-96b9-f483276ca91b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:20.424420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e218d8-259b-499b-9ee8-b70417083197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:20.425098 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e218d8-259b-499b-9ee8-b70417083197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:20.425388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e218d8-259b-499b-9ee8-b70417083197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:20.425695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e218d8-259b-499b-9ee8-b70417083197 admin admin] PATH_INFO: `/v3/projects/899fd4f1809241dcb0037945a74a0e62` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:20.427712 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e218d8-259b-499b-9ee8-b70417083197 admin admin] RBAC: Authorizing `identity:delete_project(project_id=899fd4f1809241dcb0037945a74a0e62)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:20.428909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e4df1-1391-412a-930d-71882a1340a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:20.430134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e218d8-259b-499b-9ee8-b70417083197 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:20.430893 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34e4df1-1391-412a-930d-71882a1340a6 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:20.451049 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 849/1710] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:20.453931 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4de51f-cc9b-4dea-8a22-4b9349d23247 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:20.454159 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4de51f-cc9b-4dea-8a22-4b9349d23247 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:20.454441 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4de51f-cc9b-4dea-8a22-4b9349d23247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:20.572033 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1c4de51f-cc9b-4dea-8a22-4b9349d23247 None None] MFA Rules not processed for user `afbab63bf079403b8ef652422d581d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:20.581111 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1c4de51f-cc9b-4dea-8a22-4b9349d23247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:20.593324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-39e218d8-259b-499b-9ee8-b70417083197 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '899fd4f1809241dcb0037945a74a0e62', 'request_id': 'req-39e218d8-259b-499b-9ee8-b70417083197'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:20.627787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-39e218d8-259b-499b-9ee8-b70417083197 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '899fd4f1809241dcb0037945a74a0e62', 'request_id': 'req-39e218d8-259b-499b-9ee8-b70417083197'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:20.649803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-39e218d8-259b-499b-9ee8-b70417083197 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '899fd4f1809241dcb0037945a74a0e62', 'request_id': 'req-39e218d8-259b-499b-9ee8-b70417083197'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:20.686369 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-39e218d8-259b-499b-9ee8-b70417083197 admin admin] The token cache is being invalidate because project 899fd4f1809241dcb0037945a74a0e62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:28:20.686752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-39e218d8-259b-499b-9ee8-b70417083197 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:20.689254 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-39e218d8-259b-499b-9ee8-b70417083197 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:20.699527 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 862/1711] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:28:20 2022] DELETE /identity/v3/projects/899fd4f1809241dcb0037945a74a0e62 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:20.700543 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e218d8-259b-499b-9ee8-b70417083197 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:20.706420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-39e218d8-259b-499b-9ee8-b70417083197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:20.781974 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 850/1712] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:20.783450 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd705be-cd6b-4fd3-9e52-2db336114578 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:20.783804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd705be-cd6b-4fd3-9e52-2db336114578 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:20.788472 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd705be-cd6b-4fd3-9e52-2db336114578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:20.830515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-fbd705be-cd6b-4fd3-9e52-2db336114578 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:20.831827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fbd705be-cd6b-4fd3-9e52-2db336114578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:20.880711 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e218d8-259b-499b-9ee8-b70417083197 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:20.893363 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c83452-b32c-48d3-804e-4b6ce43b5f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:20.894088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c83452-b32c-48d3-804e-4b6ce43b5f96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:20.894373 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c83452-b32c-48d3-804e-4b6ce43b5f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:20.894721 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c83452-b32c-48d3-804e-4b6ce43b5f96 admin admin] PATH_INFO: `/v3/projects/b3aaa3e46bba42d9a66c1ba8d355d4ec/users/5b4c41a94ca64d2c80f4a7522b8ff2ac/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:20.898490 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c83452-b32c-48d3-804e-4b6ce43b5f96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3aaa3e46bba42d9a66c1ba8d355d4ec, user_id=5b4c41a94ca64d2c80f4a7522b8ff2ac, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:20.903296 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c83452-b32c-48d3-804e-4b6ce43b5f96 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:20.950303 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 863/1713] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:20 2022] PUT /identity/v3/projects/b3aaa3e46bba42d9a66c1ba8d355d4ec/users/5b4c41a94ca64d2c80f4a7522b8ff2ac/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:20.951851 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb90d64-4f3a-4cc0-b91c-1b68b34c231b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:20.952439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb90d64-4f3a-4cc0-b91c-1b68b34c231b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:20.952743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb90d64-4f3a-4cc0-b91c-1b68b34c231b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:20.988984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-fbb90d64-4f3a-4cc0-b91c-1b68b34c231b None None] MFA Rules not processed for user `afbab63bf079403b8ef652422d581d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:20.994853 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fbb90d64-4f3a-4cc0-b91c-1b68b34c231b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:20.996409 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 851/1714] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:21.006245 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c0317-9037-494a-8256-0211fe7bd553 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:21.006542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c0317-9037-494a-8256-0211fe7bd553 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:21.006826 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9c0317-9037-494a-8256-0211fe7bd553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:21.053333 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:21.070848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:21.081608 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ef9c0317-9037-494a-8256-0211fe7bd553 None None] MFA Rules not processed for user `5b4c41a94ca64d2c80f4a7522b8ff2ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:21.083186 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ef9c0317-9037-494a-8256-0211fe7bd553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:21.144234 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 864/1715] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:21.146171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212e11c-a873-4e84-8ca9-23d1136ca986 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:21.146578 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212e11c-a873-4e84-8ca9-23d1136ca986 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:21.147027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212e11c-a873-4e84-8ca9-23d1136ca986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:21.174392 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8212e11c-a873-4e84-8ca9-23d1136ca986 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:21.177413 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8212e11c-a873-4e84-8ca9-23d1136ca986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:21.257702 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 852/1716] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:20 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:21.258597 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e34b7-5af0-449c-b865-d8adfd493a54 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:21.260764 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e34b7-5af0-449c-b865-d8adfd493a54 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:21.261069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e34b7-5af0-449c-b865-d8adfd493a54 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:21.263286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 853/1717] 10.222.0.39 () {54 vars in 912 bytes} [Thu Jul 28 22:28:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:28:21.272469 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e34b7-5af0-449c-b865-d8adfd493a54 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:21.277643 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-204e34b7-5af0-449c-b865-d8adfd493a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:21.292333 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 865/1718] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:21.292333 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212e11c-a873-4e84-8ca9-23d1136ca986 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:21.292333 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8212e11c-a873-4e84-8ca9-23d1136ca986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:21.458041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212e11c-a873-4e84-8ca9-23d1136ca986 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:21.471777 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f656486-c919-48aa-a841-15cc7014d33a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:21.473825 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f656486-c919-48aa-a841-15cc7014d33a service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:21.474119 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f656486-c919-48aa-a841-15cc7014d33a service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:21.474399 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f656486-c919-48aa-a841-15cc7014d33a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:21.492984 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e34b7-5af0-449c-b865-d8adfd493a54 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:21.501398 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f656486-c919-48aa-a841-15cc7014d33a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:21.505775 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f656486-c919-48aa-a841-15cc7014d33a service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:21.508602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0f369-8f45-40e2-af1a-b6c3121a1603 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:21.509499 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0f369-8f45-40e2-af1a-b6c3121a1603 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:21.512196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0f369-8f45-40e2-af1a-b6c3121a1603 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:21.512477 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0f369-8f45-40e2-af1a-b6c3121a1603 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:21.529594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 866/1719] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:21 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:21.532522 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c03d94-ab4b-4a3f-8eef-d357ab4b927a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:21.532836 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c03d94-ab4b-4a3f-8eef-d357ab4b927a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:21.533120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c03d94-ab4b-4a3f-8eef-d357ab4b927a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:21.537777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0f369-8f45-40e2-af1a-b6c3121a1603 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:21.539646 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d0f369-8f45-40e2-af1a-b6c3121a1603 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:21.565901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 854/1720] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:21.567503 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0f369-8f45-40e2-af1a-b6c3121a1603 service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:21.593343 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0f369-8f45-40e2-af1a-b6c3121a1603 service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:21.612944 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3905897-8e17-4157-926a-4d86ff9076db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:21.615647 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3905897-8e17-4157-926a-4d86ff9076db admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:21.616130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3905897-8e17-4157-926a-4d86ff9076db admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:21.616606 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3905897-8e17-4157-926a-4d86ff9076db admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:21.619479 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3905897-8e17-4157-926a-4d86ff9076db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:28:21.620129 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3905897-8e17-4157-926a-4d86ff9076db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:21.624177 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3905897-8e17-4157-926a-4d86ff9076db admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:21.647422 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d4c03d94-ab4b-4a3f-8eef-d357ab4b927a None None] MFA Rules not processed for user `09b96431ac98447283960019cadde8a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:21.648861 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d4c03d94-ab4b-4a3f-8eef-d357ab4b927a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:21.657357 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 855/1721] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:28:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:28:21.665024 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3905897-8e17-4157-926a-4d86ff9076db admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:21.695086 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3905897-8e17-4157-926a-4d86ff9076db admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:21.711115 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896dc4b-e50b-41a4-8ed7-c4a31a114803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:21.712173 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896dc4b-e50b-41a4-8ed7-c4a31a114803 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:21.712464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896dc4b-e50b-41a4-8ed7-c4a31a114803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:21.712741 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896dc4b-e50b-41a4-8ed7-c4a31a114803 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:21.713424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896dc4b-e50b-41a4-8ed7-c4a31a114803 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:21.721202 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896dc4b-e50b-41a4-8ed7-c4a31a114803 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:21.822403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 856/1722] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:21 2022] POST /identity/v3/projects => generated 365 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:21.825013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896dc4b-e50b-41a4-8ed7-c4a31a114803 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:21.829477 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a896dc4b-e50b-41a4-8ed7-c4a31a114803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:21.844884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 867/1723] 10.222.0.39 () {58 vars in 1009 bytes} [Thu Jul 28 22:28:21 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:21.846238 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c03d94-ab4b-4a3f-8eef-d357ab4b927a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:21.881551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c03d94-ab4b-4a3f-8eef-d357ab4b927a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:21.890442 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:22.015019 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:22.026537 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896dc4b-e50b-41a4-8ed7-c4a31a114803 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:22.041137 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ee9fa-e446-48e8-ae68-be7e9c462582 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:22.041760 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ee9fa-e446-48e8-ae68-be7e9c462582 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:22.042163 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ee9fa-e446-48e8-ae68-be7e9c462582 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:22.043052 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ee9fa-e446-48e8-ae68-be7e9c462582 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:22.050868 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c79ee9fa-e446-48e8-ae68-be7e9c462582 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:22.057841 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e65b3c-5d1f-4497-b572-aaad32bf270a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:22.058497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e65b3c-5d1f-4497-b572-aaad32bf270a admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:22.058749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e65b3c-5d1f-4497-b572-aaad32bf270a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:22.059018 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e65b3c-5d1f-4497-b572-aaad32bf270a admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:22.059867 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e65b3c-5d1f-4497-b572-aaad32bf270a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:22.062570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e65b3c-5d1f-4497-b572-aaad32bf270a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:22.133741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 868/1724] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:28:21 2022] POST /identity/v3/users => generated 283 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:22.146186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e65b3c-5d1f-4497-b572-aaad32bf270a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:22.150378 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a8e65b3c-5d1f-4497-b572-aaad32bf270a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:22.284091 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ee9fa-e446-48e8-ae68-be7e9c462582 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:22.286298 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ee9fa-e446-48e8-ae68-be7e9c462582 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:22.318965 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 857/1725] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:21 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:22.322746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ee9fa-e446-48e8-ae68-be7e9c462582 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:22.359999 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ee9fa-e446-48e8-ae68-be7e9c462582 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:22.391863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e98d75-78e0-4210-afa4-8d7d32cb75ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:22.392777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e98d75-78e0-4210-afa4-8d7d32cb75ee admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:22.393257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e98d75-78e0-4210-afa4-8d7d32cb75ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:22.395275 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e98d75-78e0-4210-afa4-8d7d32cb75ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:22.396207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e98d75-78e0-4210-afa4-8d7d32cb75ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:22.399652 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e98d75-78e0-4210-afa4-8d7d32cb75ee admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:22.438264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e65b3c-5d1f-4497-b572-aaad32bf270a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:22.440187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 858/1726] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:22.441729 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75005dbd-1cd6-416e-bb8e-c966830d85f3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:22.447594 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75005dbd-1cd6-416e-bb8e-c966830d85f3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:22.448085 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75005dbd-1cd6-416e-bb8e-c966830d85f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:22.454020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8259e3f-43a1-4986-8492-913e9cc4c034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:22.455824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8259e3f-43a1-4986-8492-913e9cc4c034 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:22.460142 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8259e3f-43a1-4986-8492-913e9cc4c034 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:22.460743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8259e3f-43a1-4986-8492-913e9cc4c034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:22.494920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8259e3f-43a1-4986-8492-913e9cc4c034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:22.496410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8259e3f-43a1-4986-8492-913e9cc4c034 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:22.512388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-75005dbd-1cd6-416e-bb8e-c966830d85f3 None None] MFA Rules not processed for user `7476eccb0a6f4195936bec75f7f80c7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:22.517063 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-75005dbd-1cd6-416e-bb8e-c966830d85f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:22.527842 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 869/1727] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:22.528600 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8259e3f-43a1-4986-8492-913e9cc4c034 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:22.555759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8259e3f-43a1-4986-8492-913e9cc4c034 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:22.573635 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af400e2b-50f9-405a-9d92-d9e20384270c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:22.575447 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af400e2b-50f9-405a-9d92-d9e20384270c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:22.575794 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af400e2b-50f9-405a-9d92-d9e20384270c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:22.576133 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af400e2b-50f9-405a-9d92-d9e20384270c admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:22.577010 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af400e2b-50f9-405a-9d92-d9e20384270c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:22.578829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af400e2b-50f9-405a-9d92-d9e20384270c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:22.618348 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 870/1728] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:22.630231 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38cc3d-c4ce-4bc5-9a48-6004f07fc44f None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:22.630520 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38cc3d-c4ce-4bc5-9a48-6004f07fc44f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:22.630805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38cc3d-c4ce-4bc5-9a48-6004f07fc44f None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:22.633022 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 871/1729] 10.222.0.39 () {54 vars in 912 bytes} [Thu Jul 28 22:28:22 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:28:22.648069 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c38cc3d-c4ce-4bc5-9a48-6004f07fc44f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:22.678941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 859/1730] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:22.680405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d98cd-7c80-4784-aae3-1158cdfc22c4 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:22.680684 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d98cd-7c80-4784-aae3-1158cdfc22c4 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:22.680959 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d98cd-7c80-4784-aae3-1158cdfc22c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:22.689850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c38cc3d-c4ce-4bc5-9a48-6004f07fc44f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:22.703746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be50c9-e300-4c7d-a1cf-ac6c66da2d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:22.704443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be50c9-e300-4c7d-a1cf-ac6c66da2d38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:22.704730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be50c9-e300-4c7d-a1cf-ac6c66da2d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:22.705124 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be50c9-e300-4c7d-a1cf-ac6c66da2d38 admin admin] PATH_INFO: `/v3/projects/68575e2a0319485182ada99e37b71eaa/users/a61efa34fbc64f93aeaa37fb468adcc8/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:22.746545 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-633d98cd-7c80-4784-aae3-1158cdfc22c4 None None] MFA Rules not processed for user `09b96431ac98447283960019cadde8a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:22.751891 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-633d98cd-7c80-4784-aae3-1158cdfc22c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:22.758139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be50c9-e300-4c7d-a1cf-ac6c66da2d38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68575e2a0319485182ada99e37b71eaa, user_id=a61efa34fbc64f93aeaa37fb468adcc8, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:22.778113 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be50c9-e300-4c7d-a1cf-ac6c66da2d38 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:22.797556 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 860/1731] 10.222.0.39 () {58 vars in 1009 bytes} [Thu Jul 28 22:28:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:22.831476 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 872/1732] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:22 2022] PUT /identity/v3/projects/68575e2a0319485182ada99e37b71eaa/users/a61efa34fbc64f93aeaa37fb468adcc8/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:22.843744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659a0bd-80bc-4d5a-b293-dce641fe829c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:22.845440 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659a0bd-80bc-4d5a-b293-dce641fe829c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:22.845829 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659a0bd-80bc-4d5a-b293-dce641fe829c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:22.910368 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1659a0bd-80bc-4d5a-b293-dce641fe829c None None] MFA Rules not processed for user `a61efa34fbc64f93aeaa37fb468adcc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:22.913049 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1659a0bd-80bc-4d5a-b293-dce641fe829c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:22.976423 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be50c9-e300-4c7d-a1cf-ac6c66da2d38 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:23.115659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 861/1733] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:22 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:23.120188 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659a0bd-80bc-4d5a-b293-dce641fe829c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:23.172349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be50c9-e300-4c7d-a1cf-ac6c66da2d38 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:23.182133 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06b879-8b5e-4a08-bec7-03592e5dd70c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:23.183082 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06b879-8b5e-4a08-bec7-03592e5dd70c service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:23.183489 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06b879-8b5e-4a08-bec7-03592e5dd70c service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:23.183899 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06b879-8b5e-4a08-bec7-03592e5dd70c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:23.203050 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:23.211086 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06b879-8b5e-4a08-bec7-03592e5dd70c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:23.212952 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06b879-8b5e-4a08-bec7-03592e5dd70c service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:23.286137 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 873/1734] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:23.338755 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659a0bd-80bc-4d5a-b293-dce641fe829c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:23.351782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91092c-fe09-4e2c-9d7b-f2da87b5f125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:23.352425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91092c-fe09-4e2c-9d7b-f2da87b5f125 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:23.352888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91092c-fe09-4e2c-9d7b-f2da87b5f125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:23.352982 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91092c-fe09-4e2c-9d7b-f2da87b5f125 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:23.353680 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91092c-fe09-4e2c-9d7b-f2da87b5f125 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:23.355627 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91092c-fe09-4e2c-9d7b-f2da87b5f125 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:23.435033 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06b879-8b5e-4a08-bec7-03592e5dd70c service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:23.436051 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0a06b879-8b5e-4a08-bec7-03592e5dd70c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:23.445012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 862/1735] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:23 2022] POST /identity/v3/projects => generated 363 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:23.448749 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91092c-fe09-4e2c-9d7b-f2da87b5f125 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:23.460362 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06b879-8b5e-4a08-bec7-03592e5dd70c service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:23.466827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:23.632219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91092c-fe09-4e2c-9d7b-f2da87b5f125 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:23.653839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ee6f7-f9e0-4308-a3b6-956dfefdc391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:23.654643 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ee6f7-f9e0-4308-a3b6-956dfefdc391 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:23.654933 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ee6f7-f9e0-4308-a3b6-956dfefdc391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:23.657072 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ee6f7-f9e0-4308-a3b6-956dfefdc391 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:23.662573 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ee6f7-f9e0-4308-a3b6-956dfefdc391 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:23.679382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ee6f7-f9e0-4308-a3b6-956dfefdc391 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:23.709013 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683a4fea-8cbd-4d32-ba0b-49d72ab5c4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:23.712093 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a4fea-8cbd-4d32-ba0b-49d72ab5c4c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:23.714289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a4fea-8cbd-4d32-ba0b-49d72ab5c4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:23.715416 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683a4fea-8cbd-4d32-ba0b-49d72ab5c4c1 admin admin] PATH_INFO: `/v3/users/72eefaed2c7f4151a01bfe4d80efe92f` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:23.721743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a4fea-8cbd-4d32-ba0b-49d72ab5c4c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72eefaed2c7f4151a01bfe4d80efe92f)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:23.733136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683a4fea-8cbd-4d32-ba0b-49d72ab5c4c1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:23.738786 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 863/1736] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:28:23 2022] POST /identity/v3/users => generated 288 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:28:23.742809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ee6f7-f9e0-4308-a3b6-956dfefdc391 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:23.773926 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ee6f7-f9e0-4308-a3b6-956dfefdc391 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:23.785985 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003c3cae-f4f4-47ff-bd6d-0d9afde08481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:23.786195 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c3cae-f4f4-47ff-bd6d-0d9afde08481 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:23.786451 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c3cae-f4f4-47ff-bd6d-0d9afde08481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:23.786759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003c3cae-f4f4-47ff-bd6d-0d9afde08481 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:23.787554 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c3cae-f4f4-47ff-bd6d-0d9afde08481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:23.793711 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003c3cae-f4f4-47ff-bd6d-0d9afde08481 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:23.827245 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 864/1737] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:23.828658 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bddb8-51f6-4cd3-ab5e-47bfc0ffbc54 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:23.828956 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bddb8-51f6-4cd3-ab5e-47bfc0ffbc54 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:23.829265 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bddb8-51f6-4cd3-ab5e-47bfc0ffbc54 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:23.835020 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 865/1738] 10.222.0.39 () {54 vars in 961 bytes} [Thu Jul 28 22:28:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:28:23.842138 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bddb8-51f6-4cd3-ab5e-47bfc0ffbc54 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:23.860329 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-683a4fea-8cbd-4d32-ba0b-49d72ab5c4c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72eefaed2c7f4151a01bfe4d80efe92f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:23.881085 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-683a4fea-8cbd-4d32-ba0b-49d72ab5c4c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72eefaed2c7f4151a01bfe4d80efe92f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:23.882325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bddb8-51f6-4cd3-ab5e-47bfc0ffbc54 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:23.890259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1302789-626a-429a-982b-4bcfc04968ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:23.890958 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1302789-626a-429a-982b-4bcfc04968ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:23.891244 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1302789-626a-429a-982b-4bcfc04968ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:23.891529 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1302789-626a-429a-982b-4bcfc04968ea admin admin] PATH_INFO: `/v3/projects/b30ba44848914c36aac67a876222f080/users/4aa851e2ba344f75bec09b716bd12ccd/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:23.894254 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-683a4fea-8cbd-4d32-ba0b-49d72ab5c4c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72eefaed2c7f4151a01bfe4d80efe92f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:23.896232 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-683a4fea-8cbd-4d32-ba0b-49d72ab5c4c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72eefaed2c7f4151a01bfe4d80efe92f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:23.925474 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-683a4fea-8cbd-4d32-ba0b-49d72ab5c4c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72eefaed2c7f4151a01bfe4d80efe92f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:23.929918 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1302789-626a-429a-982b-4bcfc04968ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30ba44848914c36aac67a876222f080, user_id=4aa851e2ba344f75bec09b716bd12ccd, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:23.932961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1302789-626a-429a-982b-4bcfc04968ea admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:23.934767 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 874/1739] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:28:23 2022] DELETE /identity/v3/users/72eefaed2c7f4151a01bfe4d80efe92f => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:23.936048 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f68588-049f-49ad-ac70-6b4a6c022d61 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:23.936334 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f68588-049f-49ad-ac70-6b4a6c022d61 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:23.936751 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f68588-049f-49ad-ac70-6b4a6c022d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:23.970166 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b5f68588-049f-49ad-ac70-6b4a6c022d61 None None] MFA Rules not processed for user `fe003cd7bfbe4b92b1d64d300c3e7d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:23.971168 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 866/1740] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:23 2022] PUT /identity/v3/projects/b30ba44848914c36aac67a876222f080/users/4aa851e2ba344f75bec09b716bd12ccd/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:23.971692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b5f68588-049f-49ad-ac70-6b4a6c022d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:23.974437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e47a1-64d7-4232-a4f1-430fa2e14aea None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:23.975364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e47a1-64d7-4232-a4f1-430fa2e14aea None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:23.976004 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87e47a1-64d7-4232-a4f1-430fa2e14aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:24.076484 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c87e47a1-64d7-4232-a4f1-430fa2e14aea None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:24.090733 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c87e47a1-64d7-4232-a4f1-430fa2e14aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:24.122881 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 875/1741] 10.222.0.39 () {58 vars in 1058 bytes} [Thu Jul 28 22:28:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:24.125242 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f68588-049f-49ad-ac70-6b4a6c022d61 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:24.129247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b5f68588-049f-49ad-ac70-6b4a6c022d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:24.193997 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:24.195809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:24.212914 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 867/1742] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:24.214078 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e47a1-64d7-4232-a4f1-430fa2e14aea None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:24.247121 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87e47a1-64d7-4232-a4f1-430fa2e14aea None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:24.264588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f68588-049f-49ad-ac70-6b4a6c022d61 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:24.269335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c49d11-fd15-4a90-b3e9-1950528bec59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe003cd7bfbe4b92b1d64d300c3e7d8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:24.271274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c49d11-fd15-4a90-b3e9-1950528bec59 service placement] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:24.271945 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c49d11-fd15-4a90-b3e9-1950528bec59 service placement] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:24.272216 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcdc0c-8510-45d6-b592-40dc828b6fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:24.272678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c49d11-fd15-4a90-b3e9-1950528bec59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:24.274968 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcdc0c-8510-45d6-b592-40dc828b6fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:24.275368 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-94c49d11-fd15-4a90-b3e9-1950528bec59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:24.275490 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcdc0c-8510-45d6-b592-40dc828b6fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:24.275776 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcdc0c-8510-45d6-b592-40dc828b6fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:24.276572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcdc0c-8510-45d6-b592-40dc828b6fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:24.278993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fcdc0c-8510-45d6-b592-40dc828b6fc8 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:24.295317 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 876/1743] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:24 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:24.296603 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcdc0c-8510-45d6-b592-40dc828b6fc8 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:24.298852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-07fcdc0c-8510-45d6-b592-40dc828b6fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:24.301693 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c49d11-fd15-4a90-b3e9-1950528bec59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:24.304253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c49d11-fd15-4a90-b3e9-1950528bec59 service placement] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:24.315885 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 868/1744] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:24.317763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c49d11-fd15-4a90-b3e9-1950528bec59 service placement] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:24.330348 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c49d11-fd15-4a90-b3e9-1950528bec59 service placement] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:24.337803 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c521b2-00ce-4485-9ed6-8f929c39be65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:24.338767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c521b2-00ce-4485-9ed6-8f929c39be65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:24.339234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c521b2-00ce-4485-9ed6-8f929c39be65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:24.340164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c521b2-00ce-4485-9ed6-8f929c39be65 admin admin] PATH_INFO: `/v3/projects/b30ba44848914c36aac67a876222f080/users/4aa851e2ba344f75bec09b716bd12ccd/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:24.343889 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c521b2-00ce-4485-9ed6-8f929c39be65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30ba44848914c36aac67a876222f080, user_id=4aa851e2ba344f75bec09b716bd12ccd, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:24.353449 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c521b2-00ce-4485-9ed6-8f929c39be65 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:24.380648 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 869/1745] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:24 2022] PUT /identity/v3/projects/b30ba44848914c36aac67a876222f080/users/4aa851e2ba344f75bec09b716bd12ccd/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:24.391366 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094dba9-c5dc-4b8b-9157-8cd7ebd05983 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:24.391944 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094dba9-c5dc-4b8b-9157-8cd7ebd05983 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:24.392395 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f094dba9-c5dc-4b8b-9157-8cd7ebd05983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:24.418964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fcdc0c-8510-45d6-b592-40dc828b6fc8 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:24.486725 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f094dba9-c5dc-4b8b-9157-8cd7ebd05983 None None] MFA Rules not processed for user `4aa851e2ba344f75bec09b716bd12ccd`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:24.488834 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f094dba9-c5dc-4b8b-9157-8cd7ebd05983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:24.534660 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966e5ce-a923-4891-a295-ce43d166a444 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:24.534782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966e5ce-a923-4891-a295-ce43d166a444 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:24.535704 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966e5ce-a923-4891-a295-ce43d166a444 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:24.536634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0966e5ce-a923-4891-a295-ce43d166a444 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:24.672092 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0966e5ce-a923-4891-a295-ce43d166a444 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:24.673585 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0966e5ce-a923-4891-a295-ce43d166a444 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:24.687652 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 870/1746] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:24 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:24.696591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094dba9-c5dc-4b8b-9157-8cd7ebd05983 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:24.697575 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 877/1747] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:24.698337 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f094dba9-c5dc-4b8b-9157-8cd7ebd05983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:24.884215 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f094dba9-c5dc-4b8b-9157-8cd7ebd05983 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:24.899124 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156cdd9-91f1-4fdb-9ede-aa5572c5b125 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:24.899869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156cdd9-91f1-4fdb-9ede-aa5572c5b125 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:24.904175 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156cdd9-91f1-4fdb-9ede-aa5572c5b125 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:24.904603 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156cdd9-91f1-4fdb-9ede-aa5572c5b125 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:24.918140 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156cdd9-91f1-4fdb-9ede-aa5572c5b125 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:24.919615 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156cdd9-91f1-4fdb-9ede-aa5572c5b125 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:24.933691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 871/1748] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:24 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:25.081354 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966e5ce-a923-4891-a295-ce43d166a444 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:25.099928 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0966e5ce-a923-4891-a295-ce43d166a444 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:25.112896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf964af-d90e-4721-b707-c8d6314a3242 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:25.114209 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf964af-d90e-4721-b707-c8d6314a3242 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:25.114497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf964af-d90e-4721-b707-c8d6314a3242 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:25.114795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf964af-d90e-4721-b707-c8d6314a3242 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:25.120034 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bdf964af-d90e-4721-b707-c8d6314a3242 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:25.176751 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:25.269755 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf964af-d90e-4721-b707-c8d6314a3242 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:25.271578 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf964af-d90e-4721-b707-c8d6314a3242 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:25.305581 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 878/1749] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:25 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:25.373841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad2415-b2cb-4182-86c3-106fa066f9b3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:25.375502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad2415-b2cb-4182-86c3-106fa066f9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:25.375790 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faad2415-b2cb-4182-86c3-106fa066f9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:25.454621 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-faad2415-b2cb-4182-86c3-106fa066f9b3 None None] MFA Rules not processed for user `d38b517910894f7fa4351a2120b466d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:25.461025 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-faad2415-b2cb-4182-86c3-106fa066f9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:25.692585 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 872/1750] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:25.750141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf964af-d90e-4721-b707-c8d6314a3242 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:25.751941 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bdf964af-d90e-4721-b707-c8d6314a3242 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:25.800116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf964af-d90e-4721-b707-c8d6314a3242 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:25.812019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b9e3a6-9281-4d0c-8ac8-e4357a4a3183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:25.813797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9e3a6-9281-4d0c-8ac8-e4357a4a3183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:25.815043 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9e3a6-9281-4d0c-8ac8-e4357a4a3183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:25.815525 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b9e3a6-9281-4d0c-8ac8-e4357a4a3183 admin admin] PATH_INFO: `/v3/projects/bc392ca648a74cd1a9dc1171709083ab` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:25.819240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9e3a6-9281-4d0c-8ac8-e4357a4a3183 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc392ca648a74cd1a9dc1171709083ab)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:25.822807 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b9e3a6-9281-4d0c-8ac8-e4357a4a3183 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:25.927163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-54b9e3a6-9281-4d0c-8ac8-e4357a4a3183 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc392ca648a74cd1a9dc1171709083ab', 'request_id': 'req-54b9e3a6-9281-4d0c-8ac8-e4357a4a3183'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:25.950204 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-54b9e3a6-9281-4d0c-8ac8-e4357a4a3183 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc392ca648a74cd1a9dc1171709083ab', 'request_id': 'req-54b9e3a6-9281-4d0c-8ac8-e4357a4a3183'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:26.011380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-54b9e3a6-9281-4d0c-8ac8-e4357a4a3183 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc392ca648a74cd1a9dc1171709083ab', 'request_id': 'req-54b9e3a6-9281-4d0c-8ac8-e4357a4a3183'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:26.083735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-54b9e3a6-9281-4d0c-8ac8-e4357a4a3183 admin admin] The token cache is being invalidate because project bc392ca648a74cd1a9dc1171709083ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:28:26.084032 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-54b9e3a6-9281-4d0c-8ac8-e4357a4a3183 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:26.092601 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-54b9e3a6-9281-4d0c-8ac8-e4357a4a3183 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:26.106117 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 879/1751] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:28:25 2022] DELETE /identity/v3/projects/bc392ca648a74cd1a9dc1171709083ab => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:26.110011 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad2415-b2cb-4182-86c3-106fa066f9b3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:26.135215 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faad2415-b2cb-4182-86c3-106fa066f9b3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:26.158861 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:26.271854 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b418b4-93d7-4268-9249-4a604666af36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:26.273947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b418b4-93d7-4268-9249-4a604666af36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:26.274364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b418b4-93d7-4268-9249-4a604666af36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:26.274816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b418b4-93d7-4268-9249-4a604666af36 admin admin] PATH_INFO: `/v3/users/f693151d830d4826b4694a11afd87340` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:26.279845 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b418b4-93d7-4268-9249-4a604666af36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f693151d830d4826b4694a11afd87340)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:26.282497 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b418b4-93d7-4268-9249-4a604666af36 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:26.368734 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e3b418b4-93d7-4268-9249-4a604666af36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f693151d830d4826b4694a11afd87340'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:26.379829 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e3b418b4-93d7-4268-9249-4a604666af36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f693151d830d4826b4694a11afd87340'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:26.381414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e3b418b4-93d7-4268-9249-4a604666af36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f693151d830d4826b4694a11afd87340'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:26.382242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e3b418b4-93d7-4268-9249-4a604666af36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f693151d830d4826b4694a11afd87340'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:26.404931 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e3b418b4-93d7-4268-9249-4a604666af36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f693151d830d4826b4694a11afd87340'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:26.427568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 873/1752] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:28:26 2022] DELETE /identity/v3/users/f693151d830d4826b4694a11afd87340 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:26.484352 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be2569-74e5-4008-a25c-e2964c0e9205 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:26.484965 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be2569-74e5-4008-a25c-e2964c0e9205 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:26.486869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be2569-74e5-4008-a25c-e2964c0e9205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:26.512791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f6be2569-74e5-4008-a25c-e2964c0e9205 None None] MFA Rules not processed for user `afbab63bf079403b8ef652422d581d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:26.514345 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f6be2569-74e5-4008-a25c-e2964c0e9205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:26.697899 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 880/1753] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:27.016206 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc8fed-8c72-4bd3-8272-8f13f182c72a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:27.016501 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc8fed-8c72-4bd3-8272-8f13f182c72a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:27.016781 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc8fed-8c72-4bd3-8272-8f13f182c72a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:27.097015 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-05cc8fed-8c72-4bd3-8272-8f13f182c72a None None] MFA Rules not processed for user `4ed2359ca8cb4ee18984c14a5c3646b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:27.108066 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-05cc8fed-8c72-4bd3-8272-8f13f182c72a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:27.233515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be2569-74e5-4008-a25c-e2964c0e9205 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:27.235045 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f6be2569-74e5-4008-a25c-e2964c0e9205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:27.572909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 874/1754] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 558 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:27.574419 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84307216-145c-4629-baf0-5e58ea30927b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:27.574711 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84307216-145c-4629-baf0-5e58ea30927b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:27.574989 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84307216-145c-4629-baf0-5e58ea30927b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:27.606808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:27.682771 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-84307216-145c-4629-baf0-5e58ea30927b None None] MFA Rules not processed for user `4ed2359ca8cb4ee18984c14a5c3646b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:27.686133 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6be2569-74e5-4008-a25c-e2964c0e9205 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:27.689145 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-84307216-145c-4629-baf0-5e58ea30927b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:27.707565 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca517c44-a0cc-4853-8b89-7170dc32b1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:27.722983 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca517c44-a0cc-4853-8b89-7170dc32b1a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:27.724226 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca517c44-a0cc-4853-8b89-7170dc32b1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:27.724528 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca517c44-a0cc-4853-8b89-7170dc32b1a5 admin admin] PATH_INFO: `/v3/projects/c5a06c7e3a244c9ab99f404d69a9dae9` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:27.726596 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 875/1755] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:27.738715 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca517c44-a0cc-4853-8b89-7170dc32b1a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5a06c7e3a244c9ab99f404d69a9dae9)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:27.740846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca517c44-a0cc-4853-8b89-7170dc32b1a5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:27.943069 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ca517c44-a0cc-4853-8b89-7170dc32b1a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5a06c7e3a244c9ab99f404d69a9dae9', 'request_id': 'req-ca517c44-a0cc-4853-8b89-7170dc32b1a5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:27.959590 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ca517c44-a0cc-4853-8b89-7170dc32b1a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5a06c7e3a244c9ab99f404d69a9dae9', 'request_id': 'req-ca517c44-a0cc-4853-8b89-7170dc32b1a5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:27.994683 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ca517c44-a0cc-4853-8b89-7170dc32b1a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5a06c7e3a244c9ab99f404d69a9dae9', 'request_id': 'req-ca517c44-a0cc-4853-8b89-7170dc32b1a5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:28.038233 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ca517c44-a0cc-4853-8b89-7170dc32b1a5 admin admin] The token cache is being invalidate because project c5a06c7e3a244c9ab99f404d69a9dae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:28:28.038540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ca517c44-a0cc-4853-8b89-7170dc32b1a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:28.041613 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ca517c44-a0cc-4853-8b89-7170dc32b1a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:28.050549 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 881/1756] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:28:27 2022] DELETE /identity/v3/projects/c5a06c7e3a244c9ab99f404d69a9dae9 => generated 0 bytes in 814 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:28:28.055474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf9aea-71ef-4e2a-95f4-33055f1f8fac None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:28.056279 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf9aea-71ef-4e2a-95f4-33055f1f8fac None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:28.056747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf9aea-71ef-4e2a-95f4-33055f1f8fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:28.167289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-89bf9aea-71ef-4e2a-95f4-33055f1f8fac None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:28.168554 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-89bf9aea-71ef-4e2a-95f4-33055f1f8fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:28.199031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:28.445351 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c317864-0a29-4a9a-b91c-1e3f56953fb2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:28.446081 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c317864-0a29-4a9a-b91c-1e3f56953fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:28.446642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c317864-0a29-4a9a-b91c-1e3f56953fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:28.453671 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 876/1757] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:28.459706 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc015c-cd56-4011-bdb4-697e11132022 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:28.460224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc015c-cd56-4011-bdb4-697e11132022 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:28.460730 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc015c-cd56-4011-bdb4-697e11132022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:28.550694 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d1fc015c-cd56-4011-bdb4-697e11132022 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:28.555309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d1fc015c-cd56-4011-bdb4-697e11132022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:28.598058 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 877/1758] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:28.601318 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc015c-cd56-4011-bdb4-697e11132022 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:28.643849 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc015c-cd56-4011-bdb4-697e11132022 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:28.677290 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd6177-2415-48ca-91ac-2d447fffc836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:28.678216 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd6177-2415-48ca-91ac-2d447fffc836 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:28.679238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd6177-2415-48ca-91ac-2d447fffc836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:28.683781 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd6177-2415-48ca-91ac-2d447fffc836 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:28.684853 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd6177-2415-48ca-91ac-2d447fffc836 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:28:28.685250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd6177-2415-48ca-91ac-2d447fffc836 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:28.687021 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd6177-2415-48ca-91ac-2d447fffc836 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:28.726757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-2c317864-0a29-4a9a-b91c-1e3f56953fb2 None None] MFA Rules not processed for user `85626350279c40ea99da93b9ed6994b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:28.729197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2c317864-0a29-4a9a-b91c-1e3f56953fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:28.774333 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 878/1759] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:28:28.775640 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd6177-2415-48ca-91ac-2d447fffc836 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:28.840309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd6177-2415-48ca-91ac-2d447fffc836 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:28.868740 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e726a7-eea4-419f-8aa8-7b5d91825590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:28.871651 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e726a7-eea4-419f-8aa8-7b5d91825590 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:28.880247 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e726a7-eea4-419f-8aa8-7b5d91825590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:28.881046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e726a7-eea4-419f-8aa8-7b5d91825590 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:28.882973 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e726a7-eea4-419f-8aa8-7b5d91825590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:28.887562 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e726a7-eea4-419f-8aa8-7b5d91825590 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:28.949146 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 882/1760] 10.222.0.39 () {58 vars in 1009 bytes} [Thu Jul 28 22:28:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 505 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:28:29.041455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 879/1761] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:28 2022] POST /identity/v3/projects => generated 381 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:29.043620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c317864-0a29-4a9a-b91c-1e3f56953fb2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:29.067546 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c317864-0a29-4a9a-b91c-1e3f56953fb2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:29.275389 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:29.290714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e726a7-eea4-419f-8aa8-7b5d91825590 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:29.292711 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-28e726a7-eea4-419f-8aa8-7b5d91825590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:29.348620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4711d-4959-4d31-bc69-1a2a382eca92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:29.349117 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4711d-4959-4d31-bc69-1a2a382eca92 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:29.349356 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4711d-4959-4d31-bc69-1a2a382eca92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:29.355424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4711d-4959-4d31-bc69-1a2a382eca92 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:29.356223 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4711d-4959-4d31-bc69-1a2a382eca92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:29.379796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a4711d-4959-4d31-bc69-1a2a382eca92 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:29.468752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 883/1762] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:28:29 2022] POST /identity/v3/users => generated 291 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:29.471709 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4711d-4959-4d31-bc69-1a2a382eca92 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:29.524797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4711d-4959-4d31-bc69-1a2a382eca92 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:29.544641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e56b4-df61-4aea-90e4-e5960c32200f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:29.547411 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e56b4-df61-4aea-90e4-e5960c32200f admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:29.549610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e56b4-df61-4aea-90e4-e5960c32200f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:29.551642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8e56b4-df61-4aea-90e4-e5960c32200f admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:29.554984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e56b4-df61-4aea-90e4-e5960c32200f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:29.574685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8e56b4-df61-4aea-90e4-e5960c32200f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:29.596240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 884/1763] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:29.597490 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e56b4-df61-4aea-90e4-e5960c32200f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:29.613494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8e56b4-df61-4aea-90e4-e5960c32200f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:29.637136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15eb434-d40b-45e4-a3f4-7d3456a999fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:29.638312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15eb434-d40b-45e4-a3f4-7d3456a999fc admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:29.640841 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15eb434-d40b-45e4-a3f4-7d3456a999fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:29.641150 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15eb434-d40b-45e4-a3f4-7d3456a999fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:29.642198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15eb434-d40b-45e4-a3f4-7d3456a999fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:29.647486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15eb434-d40b-45e4-a3f4-7d3456a999fc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:29.686301 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 885/1764] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:29.689789 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15eb434-d40b-45e4-a3f4-7d3456a999fc admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:29.719174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15eb434-d40b-45e4-a3f4-7d3456a999fc admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:29.742958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4460001-bbf0-42dc-aa22-c6f8603fb473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:29.743762 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4460001-bbf0-42dc-aa22-c6f8603fb473 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:29.744051 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4460001-bbf0-42dc-aa22-c6f8603fb473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:29.744328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4460001-bbf0-42dc-aa22-c6f8603fb473 admin admin] PATH_INFO: `/v3/projects/3c0ee5f50eb946b8ae9f9c7f7bcc5fb9/users/a87208a11ce04243883d83d22211d2b4/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:29.746595 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e726a7-eea4-419f-8aa8-7b5d91825590 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:29.760205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83957e4-d6e5-4bc6-8b12-fcf49336dc41 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9107872f147f6ab8698b67bb545af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:29.762662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83957e4-d6e5-4bc6-8b12-fcf49336dc41 service swift] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:29.763179 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83957e4-d6e5-4bc6-8b12-fcf49336dc41 service swift] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:29.763668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83957e4-d6e5-4bc6-8b12-fcf49336dc41 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:29.773507 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4460001-bbf0-42dc-aa22-c6f8603fb473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c0ee5f50eb946b8ae9f9c7f7bcc5fb9, user_id=a87208a11ce04243883d83d22211d2b4, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:29.777034 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4460001-bbf0-42dc-aa22-c6f8603fb473 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:29.813888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 886/1765] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:29 2022] PUT /identity/v3/projects/3c0ee5f50eb946b8ae9f9c7f7bcc5fb9/users/a87208a11ce04243883d83d22211d2b4/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:29.827574 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fba6a-8951-4a96-a90d-87ffd55815b1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:29.827905 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fba6a-8951-4a96-a90d-87ffd55815b1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:29.828328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fba6a-8951-4a96-a90d-87ffd55815b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:29.903305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e75fba6a-8951-4a96-a90d-87ffd55815b1 None None] MFA Rules not processed for user `a87208a11ce04243883d83d22211d2b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:29.909471 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:29.912381 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e75fba6a-8951-4a96-a90d-87ffd55815b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:29.950880 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83957e4-d6e5-4bc6-8b12-fcf49336dc41 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:29.953143 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83957e4-d6e5-4bc6-8b12-fcf49336dc41 service swift] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:29.964655 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 880/1766] 10.222.0.39 () {60 vars in 1384 bytes} [Thu Jul 28 22:28:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 22:28:30.167430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 887/1767] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:29 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:30.172028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83957e4-d6e5-4bc6-8b12-fcf49336dc41 service swift] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:30.198375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83957e4-d6e5-4bc6-8b12-fcf49336dc41 service swift] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:30.487559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce297b7d-3e5a-468d-a038-f3892a1e7c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:30.488861 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce297b7d-3e5a-468d-a038-f3892a1e7c73 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:30.490487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce297b7d-3e5a-468d-a038-f3892a1e7c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:30.490886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce297b7d-3e5a-468d-a038-f3892a1e7c73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:30.493977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce297b7d-3e5a-468d-a038-f3892a1e7c73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:30.510189 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce297b7d-3e5a-468d-a038-f3892a1e7c73 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:30.619957 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 881/1768] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:30 2022] POST /identity/v3/projects => generated 381 bytes in 448 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:30.623815 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fba6a-8951-4a96-a90d-87ffd55815b1 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:30.647415 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fba6a-8951-4a96-a90d-87ffd55815b1 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:30.965653 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d7280-1776-4bb3-ad10-bef15df652dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:30.967741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d7280-1776-4bb3-ad10-bef15df652dc admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:30.968046 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d7280-1776-4bb3-ad10-bef15df652dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:30.968328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d7280-1776-4bb3-ad10-bef15df652dc admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:30.969033 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d7280-1776-4bb3-ad10-bef15df652dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:30.971050 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d7280-1776-4bb3-ad10-bef15df652dc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:31.042490 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 888/1769] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:28:30 2022] POST /identity/v3/users => generated 297 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:28:31.046734 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce297b7d-3e5a-468d-a038-f3892a1e7c73 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:31.058595 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce297b7d-3e5a-468d-a038-f3892a1e7c73 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:31.095420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c14a5f-e7c6-404f-bf1c-9b9d4f3a8004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:31.096005 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c14a5f-e7c6-404f-bf1c-9b9d4f3a8004 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:31.096383 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c14a5f-e7c6-404f-bf1c-9b9d4f3a8004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:31.096678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c14a5f-e7c6-404f-bf1c-9b9d4f3a8004 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:31.097464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c14a5f-e7c6-404f-bf1c-9b9d4f3a8004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:31.099085 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c14a5f-e7c6-404f-bf1c-9b9d4f3a8004 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:31.147715 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d7280-1776-4bb3-ad10-bef15df652dc admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:31.149503 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5c8d7280-1776-4bb3-ad10-bef15df652dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:31.209968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 882/1770] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:28:31.211870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c14a5f-e7c6-404f-bf1c-9b9d4f3a8004 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:31.233031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:31.247373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c14a5f-e7c6-404f-bf1c-9b9d4f3a8004 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:31.264827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b664a0-4292-4333-b018-9edf61a4bd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:31.265713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b664a0-4292-4333-b018-9edf61a4bd50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:31.265847 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b664a0-4292-4333-b018-9edf61a4bd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:31.266093 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b664a0-4292-4333-b018-9edf61a4bd50 admin admin] PATH_INFO: `/v3/projects/a47de94473f74ae8a51811b86664470c/users/825a021fb2a94b0da4d533480affe2ad/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:31.321946 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b664a0-4292-4333-b018-9edf61a4bd50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a47de94473f74ae8a51811b86664470c, user_id=825a021fb2a94b0da4d533480affe2ad, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:31.324791 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b664a0-4292-4333-b018-9edf61a4bd50 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:31.373626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 883/1771] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:31 2022] PUT /identity/v3/projects/a47de94473f74ae8a51811b86664470c/users/825a021fb2a94b0da4d533480affe2ad/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:31.377766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b664a0-4292-4333-b018-9edf61a4bd50 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:31.426281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d7280-1776-4bb3-ad10-bef15df652dc admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:31.612620 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b664a0-4292-4333-b018-9edf61a4bd50 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:31.638961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc972d-e5d2-4d30-b163-dae64de349f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:31.645723 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc972d-e5d2-4d30-b163-dae64de349f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:31.646288 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc972d-e5d2-4d30-b163-dae64de349f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:31.646745 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc972d-e5d2-4d30-b163-dae64de349f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:31.647652 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc972d-e5d2-4d30-b163-dae64de349f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:31.649516 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc972d-e5d2-4d30-b163-dae64de349f4 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:31.680392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 884/1772] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:31.681431 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc972d-e5d2-4d30-b163-dae64de349f4 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:31.690131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3d8ff-1fa8-41d2-b867-81e420c55af8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:31.690786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3d8ff-1fa8-41d2-b867-81e420c55af8 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:31.699663 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3d8ff-1fa8-41d2-b867-81e420c55af8 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:31.699945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d3d8ff-1fa8-41d2-b867-81e420c55af8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:31.704528 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc972d-e5d2-4d30-b163-dae64de349f4 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:31.709232 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-77d3d8ff-1fa8-41d2-b867-81e420c55af8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:31.717481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec11dc-fcbe-49bf-a004-b50d037066ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:31.718537 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec11dc-fcbe-49bf-a004-b50d037066ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:31.719024 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec11dc-fcbe-49bf-a004-b50d037066ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:31.719497 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec11dc-fcbe-49bf-a004-b50d037066ac admin admin] PATH_INFO: `/v3/projects/a47de94473f74ae8a51811b86664470c/users/825a021fb2a94b0da4d533480affe2ad/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:31.721769 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec11dc-fcbe-49bf-a004-b50d037066ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=a47de94473f74ae8a51811b86664470c, user_id=825a021fb2a94b0da4d533480affe2ad, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:31.732171 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec11dc-fcbe-49bf-a004-b50d037066ac admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:31.757951 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 885/1773] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:31 2022] PUT /identity/v3/projects/a47de94473f74ae8a51811b86664470c/users/825a021fb2a94b0da4d533480affe2ad/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:31.772109 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac08836-bd10-49e4-92ac-99790fddb92d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:31.786550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac08836-bd10-49e4-92ac-99790fddb92d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:31.786929 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac08836-bd10-49e4-92ac-99790fddb92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:31.911072 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8ac08836-bd10-49e4-92ac-99790fddb92d None None] MFA Rules not processed for user `825a021fb2a94b0da4d533480affe2ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:31.914614 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8ac08836-bd10-49e4-92ac-99790fddb92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:31.934495 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3d8ff-1fa8-41d2-b867-81e420c55af8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:31.936147 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d3d8ff-1fa8-41d2-b867-81e420c55af8 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:31.951357 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:31.994452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 889/1774] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 847 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:32.177577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 886/1775] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:31 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:32.203539 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3d8ff-1fa8-41d2-b867-81e420c55af8 service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:32.205272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-77d3d8ff-1fa8-41d2-b867-81e420c55af8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:32.380942 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d3d8ff-1fa8-41d2-b867-81e420c55af8 service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:32.387901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:32.488439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7a4da-a23d-4044-92c6-80a695999506 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:32.489268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7a4da-a23d-4044-92c6-80a695999506 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:32.489628 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7a4da-a23d-4044-92c6-80a695999506 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:32.490402 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7a4da-a23d-4044-92c6-80a695999506 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:32.506776 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7a4da-a23d-4044-92c6-80a695999506 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:32.509558 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab7a4da-a23d-4044-92c6-80a695999506 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:32.532426 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 890/1776] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:33.349151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac08836-bd10-49e4-92ac-99790fddb92d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:33.363048 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac08836-bd10-49e4-92ac-99790fddb92d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:33.372800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e34e1-bbd2-42bc-9633-b5c608c11115 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825a021fb2a94b0da4d533480affe2ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47de94473f74ae8a51811b86664470c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:33.376374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e34e1-bbd2-42bc-9633-b5c608c11115 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:33.378924 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e34e1-bbd2-42bc-9633-b5c608c11115 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:33.379296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e34e1-bbd2-42bc-9633-b5c608c11115 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:33.381849 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1e34e1-bbd2-42bc-9633-b5c608c11115 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:28:33.382170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1e34e1-bbd2-42bc-9633-b5c608c11115 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:33.393057 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1e34e1-bbd2-42bc-9633-b5c608c11115 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:33.453403 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7a4da-a23d-4044-92c6-80a695999506 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:33.459576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 887/1777] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:28:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:28:33.465763 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7a4da-a23d-4044-92c6-80a695999506 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:33.476237 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cde573-45f6-4f8e-bd9a-48bf2877106b tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825a021fb2a94b0da4d533480affe2ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47de94473f74ae8a51811b86664470c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:33.503231 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cde573-45f6-4f8e-bd9a-48bf2877106b tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:33.505317 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cde573-45f6-4f8e-bd9a-48bf2877106b tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:33.507119 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cde573-45f6-4f8e-bd9a-48bf2877106b tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:33.507921 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cde573-45f6-4f8e-bd9a-48bf2877106b tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:33.510134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cde573-45f6-4f8e-bd9a-48bf2877106b tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:33.587865 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 891/1778] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:33 2022] POST /identity/v3/projects => generated 354 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:28:33.588403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e34e1-bbd2-42bc-9633-b5c608c11115 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:33.606059 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e34e1-bbd2-42bc-9633-b5c608c11115 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:33.656546 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454726c-54ef-4f67-9487-15574c4d1bea None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:33.656842 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454726c-54ef-4f67-9487-15574c4d1bea None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:33.657125 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454726c-54ef-4f67-9487-15574c4d1bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:33.711239 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1454726c-54ef-4f67-9487-15574c4d1bea None None] MFA Rules not processed for user `09b96431ac98447283960019cadde8a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:33.712598 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1454726c-54ef-4f67-9487-15574c4d1bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:33.885320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb546b-bf85-429c-bd1c-341a92217355 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825a021fb2a94b0da4d533480affe2ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47de94473f74ae8a51811b86664470c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:33.885998 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb546b-bf85-429c-bd1c-341a92217355 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:33.886290 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb546b-bf85-429c-bd1c-341a92217355 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:33.886575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb546b-bf85-429c-bd1c-341a92217355 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:33.887692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb546b-bf85-429c-bd1c-341a92217355 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:28:33.888008 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb546b-bf85-429c-bd1c-341a92217355 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:33.889655 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb546b-bf85-429c-bd1c-341a92217355 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:33.939227 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 892/1779] 10.222.0.58 () {58 vars in 1015 bytes} [Thu Jul 28 22:28:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:33.998508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 888/1780] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:28:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:28:34.014783 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454726c-54ef-4f67-9487-15574c4d1bea None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:34.022710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1454726c-54ef-4f67-9487-15574c4d1bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:34.044175 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454726c-54ef-4f67-9487-15574c4d1bea None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:34.070121 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1e1ce-4f64-400d-9cd8-45f631582f57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:34.070121 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1e1ce-4f64-400d-9cd8-45f631582f57 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:34.070121 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1e1ce-4f64-400d-9cd8-45f631582f57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:34.070121 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1e1ce-4f64-400d-9cd8-45f631582f57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:34.081034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb546b-bf85-429c-bd1c-341a92217355 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:34.085540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bdeb546b-bf85-429c-bd1c-341a92217355 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:34.114676 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c1e1ce-4f64-400d-9cd8-45f631582f57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:34.117291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c1e1ce-4f64-400d-9cd8-45f631582f57 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:34.152112 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb546b-bf85-429c-bd1c-341a92217355 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:34.165615 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145d83e-4f34-4f90-b34d-0f01a231b850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:34.166746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145d83e-4f34-4f90-b34d-0f01a231b850 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:34.167041 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145d83e-4f34-4f90-b34d-0f01a231b850 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:34.167305 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e145d83e-4f34-4f90-b34d-0f01a231b850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:34.202955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 893/1781] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:34.204066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1e1ce-4f64-400d-9cd8-45f631582f57 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:34.224638 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145d83e-4f34-4f90-b34d-0f01a231b850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:34.232770 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e145d83e-4f34-4f90-b34d-0f01a231b850 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:34.254616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1e1ce-4f64-400d-9cd8-45f631582f57 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:34.267112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f35cd6-8ea5-4d4e-a1a8-e337bad7ca75 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '825a021fb2a94b0da4d533480affe2ad', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a47de94473f74ae8a51811b86664470c', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:34.272182 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f35cd6-8ea5-4d4e-a1a8-e337bad7ca75 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:34.273135 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f35cd6-8ea5-4d4e-a1a8-e337bad7ca75 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:34.282063 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f35cd6-8ea5-4d4e-a1a8-e337bad7ca75 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] PATH_INFO: `/v3/projects/20ed6826e12a4a0195869f9e5254aa0c` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:34.283499 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f35cd6-8ea5-4d4e-a1a8-e337bad7ca75 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] RBAC: Authorizing `identity:delete_project(project_id=20ed6826e12a4a0195869f9e5254aa0c)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:34.287383 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f35cd6-8ea5-4d4e-a1a8-e337bad7ca75 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:34.315164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 889/1782] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:34.551224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f4f35cd6-8ea5-4d4e-a1a8-e337bad7ca75 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20ed6826e12a4a0195869f9e5254aa0c', 'request_id': 'req-f4f35cd6-8ea5-4d4e-a1a8-e337bad7ca75'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:34.572672 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f4f35cd6-8ea5-4d4e-a1a8-e337bad7ca75 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20ed6826e12a4a0195869f9e5254aa0c', 'request_id': 'req-f4f35cd6-8ea5-4d4e-a1a8-e337bad7ca75'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:34.590301 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f4f35cd6-8ea5-4d4e-a1a8-e337bad7ca75 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20ed6826e12a4a0195869f9e5254aa0c', 'request_id': 'req-f4f35cd6-8ea5-4d4e-a1a8-e337bad7ca75'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:34.607679 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f4f35cd6-8ea5-4d4e-a1a8-e337bad7ca75 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] The token cache is being invalidate because project 20ed6826e12a4a0195869f9e5254aa0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:28:34.607991 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f4f35cd6-8ea5-4d4e-a1a8-e337bad7ca75 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:34.608984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f4f35cd6-8ea5-4d4e-a1a8-e337bad7ca75 tempest-VolumeQuotaClassesTest-1749197800 tempest-VolumeQuotaClassesTest-1749197800-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:34.616219 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 894/1783] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:28:34 2022] DELETE /identity/v3/projects/20ed6826e12a4a0195869f9e5254aa0c => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:35.865952 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145d83e-4f34-4f90-b34d-0f01a231b850 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:36.220626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e145d83e-4f34-4f90-b34d-0f01a231b850 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:36.286254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:36.449821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01129bcf-8789-4d42-878a-f7ab95c6a165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:36.450882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01129bcf-8789-4d42-878a-f7ab95c6a165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:36.451327 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01129bcf-8789-4d42-878a-f7ab95c6a165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:36.451757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01129bcf-8789-4d42-878a-f7ab95c6a165 admin admin] PATH_INFO: `/v3/users/a87208a11ce04243883d83d22211d2b4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:36.453275 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01129bcf-8789-4d42-878a-f7ab95c6a165 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a87208a11ce04243883d83d22211d2b4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:36.455458 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01129bcf-8789-4d42-878a-f7ab95c6a165 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:36.574864 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-01129bcf-8789-4d42-878a-f7ab95c6a165 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a87208a11ce04243883d83d22211d2b4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:36.588117 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-01129bcf-8789-4d42-878a-f7ab95c6a165 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a87208a11ce04243883d83d22211d2b4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:36.588879 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-01129bcf-8789-4d42-878a-f7ab95c6a165 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a87208a11ce04243883d83d22211d2b4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:36.589509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-01129bcf-8789-4d42-878a-f7ab95c6a165 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a87208a11ce04243883d83d22211d2b4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:36.653370 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-01129bcf-8789-4d42-878a-f7ab95c6a165 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a87208a11ce04243883d83d22211d2b4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:36.688705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 890/1784] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:28:35 2022] DELETE /identity/v3/users/a87208a11ce04243883d83d22211d2b4 => generated 0 bytes in 822 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:36.692442 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef33d13-10c0-4f95-8b24-a51fa40f51b1 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:36.692978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef33d13-10c0-4f95-8b24-a51fa40f51b1 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:36.693174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef33d13-10c0-4f95-8b24-a51fa40f51b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:36.721887 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:36.741315 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-bef33d13-10c0-4f95-8b24-a51fa40f51b1 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:36.742628 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bef33d13-10c0-4f95-8b24-a51fa40f51b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:36.929264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 895/1785] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:36.954050 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01129bcf-8789-4d42-878a-f7ab95c6a165 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:36.955481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-01129bcf-8789-4d42-878a-f7ab95c6a165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:37.069713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01129bcf-8789-4d42-878a-f7ab95c6a165 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:37.074787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2b0e1-8ad8-452c-8ec5-bf574a427878 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:37.075888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2b0e1-8ad8-452c-8ec5-bf574a427878 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:37.076350 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2b0e1-8ad8-452c-8ec5-bf574a427878 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:37.076630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2b0e1-8ad8-452c-8ec5-bf574a427878 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:37.104125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2b0e1-8ad8-452c-8ec5-bf574a427878 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:37.106987 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2b0e1-8ad8-452c-8ec5-bf574a427878 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:37.120096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 891/1786] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:38.831828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40834433-08e9-4c35-b1ba-31d6ba9c9922 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:38.832388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40834433-08e9-4c35-b1ba-31d6ba9c9922 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:38.833832 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40834433-08e9-4c35-b1ba-31d6ba9c9922 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:38.835348 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 896/1787] 10.222.0.39 () {54 vars in 921 bytes} [Thu Jul 28 22:28:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:28:38.965215 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0680f37-36f8-420f-887b-d17b5cff2425 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:38.965825 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0680f37-36f8-420f-887b-d17b5cff2425 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:38.965916 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0680f37-36f8-420f-887b-d17b5cff2425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:39.032782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a0680f37-36f8-420f-887b-d17b5cff2425 None None] MFA Rules not processed for user `77efe8046ea0468cb0c7c58c24fc4e75`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:39.037015 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a0680f37-36f8-420f-887b-d17b5cff2425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:39.105483 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:39.157492 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 892/1788] 10.222.0.39 () {58 vars in 1018 bytes} [Thu Jul 28 22:28:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:39.875110 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40834433-08e9-4c35-b1ba-31d6ba9c9922 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:39.881107 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-40834433-08e9-4c35-b1ba-31d6ba9c9922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:39.941302 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40834433-08e9-4c35-b1ba-31d6ba9c9922 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:39.957838 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0680f37-36f8-420f-887b-d17b5cff2425 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:39.959589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a0680f37-36f8-420f-887b-d17b5cff2425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:39.963527 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496721fd-36ba-4200-aa7c-70d9932d9974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:39.966464 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496721fd-36ba-4200-aa7c-70d9932d9974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:39.966995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496721fd-36ba-4200-aa7c-70d9932d9974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:39.967322 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496721fd-36ba-4200-aa7c-70d9932d9974 admin admin] PATH_INFO: `/v3/users/a61efa34fbc64f93aeaa37fb468adcc8` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:39.970486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496721fd-36ba-4200-aa7c-70d9932d9974 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a61efa34fbc64f93aeaa37fb468adcc8)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:39.975609 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496721fd-36ba-4200-aa7c-70d9932d9974 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:39.998124 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0680f37-36f8-420f-887b-d17b5cff2425 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:40.023194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92df46-efb0-4aea-826b-e2543f6b71ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:40.023984 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92df46-efb0-4aea-826b-e2543f6b71ea service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:40.025742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92df46-efb0-4aea-826b-e2543f6b71ea service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:40.026345 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92df46-efb0-4aea-826b-e2543f6b71ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:40.056086 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92df46-efb0-4aea-826b-e2543f6b71ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:40.059566 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92df46-efb0-4aea-826b-e2543f6b71ea service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:40.076672 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-496721fd-36ba-4200-aa7c-70d9932d9974 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a61efa34fbc64f93aeaa37fb468adcc8'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:40.092256 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-496721fd-36ba-4200-aa7c-70d9932d9974 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a61efa34fbc64f93aeaa37fb468adcc8'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:40.105896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-496721fd-36ba-4200-aa7c-70d9932d9974 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a61efa34fbc64f93aeaa37fb468adcc8'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:40.106918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-496721fd-36ba-4200-aa7c-70d9932d9974 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a61efa34fbc64f93aeaa37fb468adcc8'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:40.107718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 893/1789] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:28:40.147748 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-496721fd-36ba-4200-aa7c-70d9932d9974 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a61efa34fbc64f93aeaa37fb468adcc8'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:40.162879 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 897/1790] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:28:39 2022] DELETE /identity/v3/users/a61efa34fbc64f93aeaa37fb468adcc8 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:40.167728 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb10bf-385b-4982-90ec-54e851ff666a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:40.168028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb10bf-385b-4982-90ec-54e851ff666a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:40.168358 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb10bf-385b-4982-90ec-54e851ff666a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:40.272957 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b6bb10bf-385b-4982-90ec-54e851ff666a None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:40.274259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b6bb10bf-385b-4982-90ec-54e851ff666a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:40.415082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 894/1791] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:40.649608 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496721fd-36ba-4200-aa7c-70d9932d9974 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:40.656035 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-496721fd-36ba-4200-aa7c-70d9932d9974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:40.793383 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496721fd-36ba-4200-aa7c-70d9932d9974 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:40.804901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a64318-90c1-4935-b375-bcc9c2526394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:40.805649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a64318-90c1-4935-b375-bcc9c2526394 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:40.806005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a64318-90c1-4935-b375-bcc9c2526394 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:40.806282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a64318-90c1-4935-b375-bcc9c2526394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:40.845560 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a64318-90c1-4935-b375-bcc9c2526394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:40.850705 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a64318-90c1-4935-b375-bcc9c2526394 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:40.929647 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 898/1792] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:41.502320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582a7b7-a51e-4f36-bed7-3b647dd5e580 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:41.504122 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582a7b7-a51e-4f36-bed7-3b647dd5e580 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:41.505232 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582a7b7-a51e-4f36-bed7-3b647dd5e580 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:41.510217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 895/1793] 10.222.0.39 () {54 vars in 921 bytes} [Thu Jul 28 22:28:41 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:28:41.696667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b902ac2-d40a-424c-9191-b43bb17a868a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:41.696961 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b902ac2-d40a-424c-9191-b43bb17a868a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:41.697236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b902ac2-d40a-424c-9191-b43bb17a868a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:41.737928 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-2b902ac2-d40a-424c-9191-b43bb17a868a None None] MFA Rules not processed for user `77efe8046ea0468cb0c7c58c24fc4e75`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:41.740167 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2b902ac2-d40a-424c-9191-b43bb17a868a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:41.782123 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:41.983148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 899/1794] 10.222.0.39 () {58 vars in 1018 bytes} [Thu Jul 28 22:28:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:42.272640 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582a7b7-a51e-4f36-bed7-3b647dd5e580 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:42.274467 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0582a7b7-a51e-4f36-bed7-3b647dd5e580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:42.339769 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0582a7b7-a51e-4f36-bed7-3b647dd5e580 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:42.358327 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d76f6-af5a-445e-be45-28a0af842824 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:42.359532 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d76f6-af5a-445e-be45-28a0af842824 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:42.360140 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d76f6-af5a-445e-be45-28a0af842824 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:42.360756 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d76f6-af5a-445e-be45-28a0af842824 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:42.380696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d76f6-af5a-445e-be45-28a0af842824 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:42.384078 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d76f6-af5a-445e-be45-28a0af842824 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:42.426841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 896/1795] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:42.809825 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b902ac2-d40a-424c-9191-b43bb17a868a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:42.812177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2b902ac2-d40a-424c-9191-b43bb17a868a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:42.843140 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b902ac2-d40a-424c-9191-b43bb17a868a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:42.867947 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2f0dd-2eae-4812-946d-e254621854d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:42.868623 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2f0dd-2eae-4812-946d-e254621854d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:42.868909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2f0dd-2eae-4812-946d-e254621854d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:42.869196 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2f0dd-2eae-4812-946d-e254621854d7 admin admin] PATH_INFO: `/v3/projects/3c0ee5f50eb946b8ae9f9c7f7bcc5fb9` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:42.878822 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a2f0dd-2eae-4812-946d-e254621854d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c0ee5f50eb946b8ae9f9c7f7bcc5fb9)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:42.882375 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a2f0dd-2eae-4812-946d-e254621854d7 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:43.040537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c1a2f0dd-2eae-4812-946d-e254621854d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c0ee5f50eb946b8ae9f9c7f7bcc5fb9', 'request_id': 'req-c1a2f0dd-2eae-4812-946d-e254621854d7'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:43.085861 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c1a2f0dd-2eae-4812-946d-e254621854d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c0ee5f50eb946b8ae9f9c7f7bcc5fb9', 'request_id': 'req-c1a2f0dd-2eae-4812-946d-e254621854d7'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:43.104441 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c1a2f0dd-2eae-4812-946d-e254621854d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c0ee5f50eb946b8ae9f9c7f7bcc5fb9', 'request_id': 'req-c1a2f0dd-2eae-4812-946d-e254621854d7'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:43.141102 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c1a2f0dd-2eae-4812-946d-e254621854d7 admin admin] The token cache is being invalidate because project 3c0ee5f50eb946b8ae9f9c7f7bcc5fb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:28:43.142473 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c1a2f0dd-2eae-4812-946d-e254621854d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:43.146222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c1a2f0dd-2eae-4812-946d-e254621854d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:43.150518 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 900/1796] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:28:42 2022] DELETE /identity/v3/projects/3c0ee5f50eb946b8ae9f9c7f7bcc5fb9 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:43.155716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d76f6-af5a-445e-be45-28a0af842824 service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:43.189506 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d76f6-af5a-445e-be45-28a0af842824 service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:43.375683 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a5c38-3108-4657-ad49-23a10acb61cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:43.376845 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a5c38-3108-4657-ad49-23a10acb61cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:43.377290 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a5c38-3108-4657-ad49-23a10acb61cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:43.377822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a5c38-3108-4657-ad49-23a10acb61cb admin admin] PATH_INFO: `/v3/users/825a021fb2a94b0da4d533480affe2ad` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:43.381167 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a5c38-3108-4657-ad49-23a10acb61cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=825a021fb2a94b0da4d533480affe2ad)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:43.384314 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a5c38-3108-4657-ad49-23a10acb61cb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:43.471940 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-325a5c38-3108-4657-ad49-23a10acb61cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '825a021fb2a94b0da4d533480affe2ad'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:43.484859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:43.493538 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-325a5c38-3108-4657-ad49-23a10acb61cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '825a021fb2a94b0da4d533480affe2ad'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:43.497406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-325a5c38-3108-4657-ad49-23a10acb61cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '825a021fb2a94b0da4d533480affe2ad'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:43.498476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-325a5c38-3108-4657-ad49-23a10acb61cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '825a021fb2a94b0da4d533480affe2ad'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:43.535507 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-325a5c38-3108-4657-ad49-23a10acb61cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '825a021fb2a94b0da4d533480affe2ad'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:43.570702 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 897/1797] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:28:43 2022] DELETE /identity/v3/users/825a021fb2a94b0da4d533480affe2ad => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:28:45.028560 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2f0dd-2eae-4812-946d-e254621854d7 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:45.031202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c1a2f0dd-2eae-4812-946d-e254621854d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:45.061241 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2f0dd-2eae-4812-946d-e254621854d7 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:45.179996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa217bb-2cef-4bed-bac1-9721ca7e79e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:45.180875 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa217bb-2cef-4bed-bac1-9721ca7e79e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:45.181345 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa217bb-2cef-4bed-bac1-9721ca7e79e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:45.181759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa217bb-2cef-4bed-bac1-9721ca7e79e3 admin admin] PATH_INFO: `/v3/projects/68575e2a0319485182ada99e37b71eaa` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:45.186008 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa217bb-2cef-4bed-bac1-9721ca7e79e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68575e2a0319485182ada99e37b71eaa)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:45.186008 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa217bb-2cef-4bed-bac1-9721ca7e79e3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:45.257777 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:45.274780 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7aa217bb-2cef-4bed-bac1-9721ca7e79e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68575e2a0319485182ada99e37b71eaa', 'request_id': 'req-7aa217bb-2cef-4bed-bac1-9721ca7e79e3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:45.295556 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7aa217bb-2cef-4bed-bac1-9721ca7e79e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68575e2a0319485182ada99e37b71eaa', 'request_id': 'req-7aa217bb-2cef-4bed-bac1-9721ca7e79e3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:45.303449 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a5c38-3108-4657-ad49-23a10acb61cb admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:45.305026 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-325a5c38-3108-4657-ad49-23a10acb61cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:45.334434 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7aa217bb-2cef-4bed-bac1-9721ca7e79e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68575e2a0319485182ada99e37b71eaa', 'request_id': 'req-7aa217bb-2cef-4bed-bac1-9721ca7e79e3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:45.336864 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a5c38-3108-4657-ad49-23a10acb61cb admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:45.364660 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7aa217bb-2cef-4bed-bac1-9721ca7e79e3 admin admin] The token cache is being invalidate because project 68575e2a0319485182ada99e37b71eaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:28:45.364965 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7aa217bb-2cef-4bed-bac1-9721ca7e79e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:45.368758 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7aa217bb-2cef-4bed-bac1-9721ca7e79e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:45.372150 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 901/1798] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:28:45 2022] DELETE /identity/v3/projects/68575e2a0319485182ada99e37b71eaa => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:45.376248 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa217bb-2cef-4bed-bac1-9721ca7e79e3 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:45.379896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7aa217bb-2cef-4bed-bac1-9721ca7e79e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:45.476096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b2fe7-e566-471b-ab81-4442008e8fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:45.476866 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b2fe7-e566-471b-ab81-4442008e8fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:45.477264 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b2fe7-e566-471b-ab81-4442008e8fee admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:45.477674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b2fe7-e566-471b-ab81-4442008e8fee admin admin] PATH_INFO: `/v3/users/d52f5056cf0e49c2945fd718f19af77e` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:45.478983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b2fe7-e566-471b-ab81-4442008e8fee admin admin] RBAC: Authorizing `identity:delete_user(user_id=d52f5056cf0e49c2945fd718f19af77e)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:45.481934 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b2fe7-e566-471b-ab81-4442008e8fee admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:45.491120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa217bb-2cef-4bed-bac1-9721ca7e79e3 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:45.498148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ec62f-85b7-4fe7-b1b8-7c8aa55f7e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:45.499096 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ec62f-85b7-4fe7-b1b8-7c8aa55f7e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:45.499611 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ec62f-85b7-4fe7-b1b8-7c8aa55f7e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:45.500031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86ec62f-85b7-4fe7-b1b8-7c8aa55f7e2a admin admin] PATH_INFO: `/v3/users/4aa851e2ba344f75bec09b716bd12ccd` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:45.520908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ec62f-85b7-4fe7-b1b8-7c8aa55f7e2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aa851e2ba344f75bec09b716bd12ccd)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:45.526957 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86ec62f-85b7-4fe7-b1b8-7c8aa55f7e2a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:45.602285 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-883b2fe7-e566-471b-ab81-4442008e8fee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd52f5056cf0e49c2945fd718f19af77e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:45.612082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-883b2fe7-e566-471b-ab81-4442008e8fee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd52f5056cf0e49c2945fd718f19af77e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:45.613295 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-883b2fe7-e566-471b-ab81-4442008e8fee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd52f5056cf0e49c2945fd718f19af77e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:45.614163 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-883b2fe7-e566-471b-ab81-4442008e8fee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd52f5056cf0e49c2945fd718f19af77e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:45.629893 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f86ec62f-85b7-4fe7-b1b8-7c8aa55f7e2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aa851e2ba344f75bec09b716bd12ccd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:45.643892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f86ec62f-85b7-4fe7-b1b8-7c8aa55f7e2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aa851e2ba344f75bec09b716bd12ccd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:45.654552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-883b2fe7-e566-471b-ab81-4442008e8fee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd52f5056cf0e49c2945fd718f19af77e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:45.659915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f86ec62f-85b7-4fe7-b1b8-7c8aa55f7e2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aa851e2ba344f75bec09b716bd12ccd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:45.661569 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f86ec62f-85b7-4fe7-b1b8-7c8aa55f7e2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aa851e2ba344f75bec09b716bd12ccd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:45.678872 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 898/1799] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:28:45 2022] DELETE /identity/v3/users/d52f5056cf0e49c2945fd718f19af77e => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:45.683509 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f86ec62f-85b7-4fe7-b1b8-7c8aa55f7e2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aa851e2ba344f75bec09b716bd12ccd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:45.685427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3eef3-0797-45e5-8298-1bfc6fdceb80 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:45.685832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3eef3-0797-45e5-8298-1bfc6fdceb80 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:45.686197 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3eef3-0797-45e5-8298-1bfc6fdceb80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:45.695026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 902/1800] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:28:45 2022] DELETE /identity/v3/users/4aa851e2ba344f75bec09b716bd12ccd => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:45.719085 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-afb3eef3-0797-45e5-8298-1bfc6fdceb80 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:45.720539 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-afb3eef3-0797-45e5-8298-1bfc6fdceb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:45.779187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:45.817626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 899/1801] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:45.825369 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ec62f-85b7-4fe7-b1b8-7c8aa55f7e2a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:45.826839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f86ec62f-85b7-4fe7-b1b8-7c8aa55f7e2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:45.910079 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:45.916594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86ec62f-85b7-4fe7-b1b8-7c8aa55f7e2a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:45.921021 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc626d-c45c-462e-af01-586a82cf4159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:45.921752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc626d-c45c-462e-af01-586a82cf4159 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:45.922043 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc626d-c45c-462e-af01-586a82cf4159 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:45.922328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cc626d-c45c-462e-af01-586a82cf4159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:45.936061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc626d-c45c-462e-af01-586a82cf4159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:45.938165 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cc626d-c45c-462e-af01-586a82cf4159 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:45.952596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 903/1802] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:46.394083 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3eef3-0797-45e5-8298-1bfc6fdceb80 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:46.394083 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-afb3eef3-0797-45e5-8298-1bfc6fdceb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:46.405608 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3eef3-0797-45e5-8298-1bfc6fdceb80 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:46.418268 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52e48c-6b03-48a0-8e31-b2ff90e39a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:46.419634 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52e48c-6b03-48a0-8e31-b2ff90e39a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:46.420019 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52e48c-6b03-48a0-8e31-b2ff90e39a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:46.420760 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52e48c-6b03-48a0-8e31-b2ff90e39a48 admin admin] PATH_INFO: `/v3/projects/a47de94473f74ae8a51811b86664470c` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:46.422469 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52e48c-6b03-48a0-8e31-b2ff90e39a48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a47de94473f74ae8a51811b86664470c)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:46.425970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad52e48c-6b03-48a0-8e31-b2ff90e39a48 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:46.510614 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc626d-c45c-462e-af01-586a82cf4159 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:46.514096 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-45cc626d-c45c-462e-af01-586a82cf4159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:46.556063 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ad52e48c-6b03-48a0-8e31-b2ff90e39a48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a47de94473f74ae8a51811b86664470c', 'request_id': 'req-ad52e48c-6b03-48a0-8e31-b2ff90e39a48'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:46.562383 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cc626d-c45c-462e-af01-586a82cf4159 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:46.578571 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ad52e48c-6b03-48a0-8e31-b2ff90e39a48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a47de94473f74ae8a51811b86664470c', 'request_id': 'req-ad52e48c-6b03-48a0-8e31-b2ff90e39a48'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:46.593118 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ad52e48c-6b03-48a0-8e31-b2ff90e39a48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a47de94473f74ae8a51811b86664470c', 'request_id': 'req-ad52e48c-6b03-48a0-8e31-b2ff90e39a48'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:46.648443 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ad52e48c-6b03-48a0-8e31-b2ff90e39a48 admin admin] The token cache is being invalidate because project a47de94473f74ae8a51811b86664470c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:28:46.649046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ad52e48c-6b03-48a0-8e31-b2ff90e39a48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:46.650137 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ad52e48c-6b03-48a0-8e31-b2ff90e39a48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:46.652347 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 900/1803] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:28:46 2022] DELETE /identity/v3/projects/a47de94473f74ae8a51811b86664470c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:46.676023 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c6f08-5846-4b21-8841-912b0f637200 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:46.676807 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c6f08-5846-4b21-8841-912b0f637200 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:46.677307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c6f08-5846-4b21-8841-912b0f637200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:46.739194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-916c6f08-5846-4b21-8841-912b0f637200 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:46.740848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-916c6f08-5846-4b21-8841-912b0f637200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:46.744545 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8892a987-f509-4a87-a477-08a77a7d4fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:46.746114 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892a987-f509-4a87-a477-08a77a7d4fd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:46.746552 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892a987-f509-4a87-a477-08a77a7d4fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:46.747230 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892a987-f509-4a87-a477-08a77a7d4fd1 admin admin] PATH_INFO: `/v3/projects/bfa5bfff9d9044198a05f52d08356011` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:46.748777 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892a987-f509-4a87-a477-08a77a7d4fd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfa5bfff9d9044198a05f52d08356011)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:46.752566 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 901/1804] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:46.753492 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892a987-f509-4a87-a477-08a77a7d4fd1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:46.765748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531193ef-7f79-4dd7-92f4-05fc322d29e3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:46.766182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531193ef-7f79-4dd7-92f4-05fc322d29e3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:46.766734 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531193ef-7f79-4dd7-92f4-05fc322d29e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:46.793045 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-531193ef-7f79-4dd7-92f4-05fc322d29e3 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:46.794572 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-531193ef-7f79-4dd7-92f4-05fc322d29e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:46.805298 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 902/1805] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:46.809209 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531193ef-7f79-4dd7-92f4-05fc322d29e3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:46.839080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8892a987-f509-4a87-a477-08a77a7d4fd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfa5bfff9d9044198a05f52d08356011', 'request_id': 'req-8892a987-f509-4a87-a477-08a77a7d4fd1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:46.841136 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531193ef-7f79-4dd7-92f4-05fc322d29e3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:46.870402 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8892a987-f509-4a87-a477-08a77a7d4fd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfa5bfff9d9044198a05f52d08356011', 'request_id': 'req-8892a987-f509-4a87-a477-08a77a7d4fd1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:46.879006 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8892a987-f509-4a87-a477-08a77a7d4fd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfa5bfff9d9044198a05f52d08356011', 'request_id': 'req-8892a987-f509-4a87-a477-08a77a7d4fd1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:46.908686 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8892a987-f509-4a87-a477-08a77a7d4fd1 admin admin] The token cache is being invalidate because project bfa5bfff9d9044198a05f52d08356011 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:28:46.909635 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8892a987-f509-4a87-a477-08a77a7d4fd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:46.910638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8892a987-f509-4a87-a477-08a77a7d4fd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:46.915376 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 904/1806] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:28:46 2022] DELETE /identity/v3/projects/bfa5bfff9d9044198a05f52d08356011 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:46.917013 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8892a987-f509-4a87-a477-08a77a7d4fd1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:46.920464 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8892a987-f509-4a87-a477-08a77a7d4fd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:46.973477 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296dba4-0230-4bb2-b8e7-ba2c6d42e0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:46.974146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296dba4-0230-4bb2-b8e7-ba2c6d42e0c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:46.974427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296dba4-0230-4bb2-b8e7-ba2c6d42e0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:46.974698 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3296dba4-0230-4bb2-b8e7-ba2c6d42e0c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:46.975525 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296dba4-0230-4bb2-b8e7-ba2c6d42e0c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:28:46.975830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296dba4-0230-4bb2-b8e7-ba2c6d42e0c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:46.980052 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3296dba4-0230-4bb2-b8e7-ba2c6d42e0c9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:47.007996 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 903/1807] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:28:47.009427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296dba4-0230-4bb2-b8e7-ba2c6d42e0c9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:47.014347 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3296dba4-0230-4bb2-b8e7-ba2c6d42e0c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:47.040644 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8892a987-f509-4a87-a477-08a77a7d4fd1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:47.049213 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3296dba4-0230-4bb2-b8e7-ba2c6d42e0c9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:47.057229 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fea7927-177f-4922-9d7e-ebf1144c4763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:47.057947 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea7927-177f-4922-9d7e-ebf1144c4763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:47.058233 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea7927-177f-4922-9d7e-ebf1144c4763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:47.058510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fea7927-177f-4922-9d7e-ebf1144c4763 admin admin] PATH_INFO: `/v3/users/5b4c41a94ca64d2c80f4a7522b8ff2ac` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:47.062382 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea7927-177f-4922-9d7e-ebf1144c4763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b4c41a94ca64d2c80f4a7522b8ff2ac)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:47.064864 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cae68-c7c2-4a5a-a336-1d8ed9d7e20d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:47.066249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cae68-c7c2-4a5a-a336-1d8ed9d7e20d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:47.066842 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cae68-c7c2-4a5a-a336-1d8ed9d7e20d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:47.067045 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452cae68-c7c2-4a5a-a336-1d8ed9d7e20d admin admin] PATH_INFO: `/v3/projects/b30ba44848914c36aac67a876222f080` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:47.072191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452cae68-c7c2-4a5a-a336-1d8ed9d7e20d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b30ba44848914c36aac67a876222f080)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:47.074165 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452cae68-c7c2-4a5a-a336-1d8ed9d7e20d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:47.074766 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fea7927-177f-4922-9d7e-ebf1144c4763 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:47.105846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:47.152176 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:47.155785 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-452cae68-c7c2-4a5a-a336-1d8ed9d7e20d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b30ba44848914c36aac67a876222f080', 'request_id': 'req-452cae68-c7c2-4a5a-a336-1d8ed9d7e20d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:47.158282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2fea7927-177f-4922-9d7e-ebf1144c4763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b4c41a94ca64d2c80f4a7522b8ff2ac'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:47.167721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-452cae68-c7c2-4a5a-a336-1d8ed9d7e20d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b30ba44848914c36aac67a876222f080', 'request_id': 'req-452cae68-c7c2-4a5a-a336-1d8ed9d7e20d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:47.174218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2fea7927-177f-4922-9d7e-ebf1144c4763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b4c41a94ca64d2c80f4a7522b8ff2ac'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:47.181253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-452cae68-c7c2-4a5a-a336-1d8ed9d7e20d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b30ba44848914c36aac67a876222f080', 'request_id': 'req-452cae68-c7c2-4a5a-a336-1d8ed9d7e20d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:47.184836 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2fea7927-177f-4922-9d7e-ebf1144c4763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b4c41a94ca64d2c80f4a7522b8ff2ac'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:47.185285 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2fea7927-177f-4922-9d7e-ebf1144c4763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b4c41a94ca64d2c80f4a7522b8ff2ac'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:47.199679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-452cae68-c7c2-4a5a-a336-1d8ed9d7e20d admin admin] The token cache is being invalidate because project b30ba44848914c36aac67a876222f080 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:28:47.200123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-452cae68-c7c2-4a5a-a336-1d8ed9d7e20d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:47.201105 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-452cae68-c7c2-4a5a-a336-1d8ed9d7e20d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:47.206757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 904/1808] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:28:47 2022] DELETE /identity/v3/projects/b30ba44848914c36aac67a876222f080 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:47.207784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cae68-c7c2-4a5a-a336-1d8ed9d7e20d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:47.209505 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-452cae68-c7c2-4a5a-a336-1d8ed9d7e20d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:47.220695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2fea7927-177f-4922-9d7e-ebf1144c4763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b4c41a94ca64d2c80f4a7522b8ff2ac'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:47.234915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 905/1809] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:28:46 2022] DELETE /identity/v3/users/5b4c41a94ca64d2c80f4a7522b8ff2ac => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:47.236610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbf690-30c4-42c1-bce1-343c5d317c5f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:47.237065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbf690-30c4-42c1-bce1-343c5d317c5f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:47.238334 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbf690-30c4-42c1-bce1-343c5d317c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:47.265424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-abbbf690-30c4-42c1-bce1-343c5d317c5f None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:47.267300 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-abbbf690-30c4-42c1-bce1-343c5d317c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:47.340974 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452cae68-c7c2-4a5a-a336-1d8ed9d7e20d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:47.401311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 906/1810] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:47.409285 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119eea0-3e14-48d4-89ca-5b735b51eb27 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:47.409285 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119eea0-3e14-48d4-89ca-5b735b51eb27 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:47.409614 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119eea0-3e14-48d4-89ca-5b735b51eb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:47.433895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9118b-e7cf-4233-a1dc-ae9132076c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:47.434667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9118b-e7cf-4233-a1dc-ae9132076c33 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:47.434902 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9118b-e7cf-4233-a1dc-ae9132076c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:47.435570 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9118b-e7cf-4233-a1dc-ae9132076c33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:47.436117 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9118b-e7cf-4233-a1dc-ae9132076c33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:47.437389 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1119eea0-3e14-48d4-89ca-5b735b51eb27 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:47.438420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9118b-e7cf-4233-a1dc-ae9132076c33 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:47.439164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1119eea0-3e14-48d4-89ca-5b735b51eb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:47.462849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 907/1811] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:47.464036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119eea0-3e14-48d4-89ca-5b735b51eb27 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:47.474101 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119eea0-3e14-48d4-89ca-5b735b51eb27 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:47.480413 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34340d-608e-4bbd-8674-35778d4f355e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:47.481285 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34340d-608e-4bbd-8674-35778d4f355e admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:47.481595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34340d-608e-4bbd-8674-35778d4f355e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:47.481872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34340d-608e-4bbd-8674-35778d4f355e admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:47.482583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34340d-608e-4bbd-8674-35778d4f355e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:28:47.482878 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34340d-608e-4bbd-8674-35778d4f355e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:47.484523 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34340d-608e-4bbd-8674-35778d4f355e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:47.489079 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 905/1812] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:47 2022] POST /identity/v3/projects => generated 385 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:47.493340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9118b-e7cf-4233-a1dc-ae9132076c33 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:47.509717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 908/1813] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:28:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:28:47.512995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34340d-608e-4bbd-8674-35778d4f355e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:47.526193 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34340d-608e-4bbd-8674-35778d4f355e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:47.572657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9118b-e7cf-4233-a1dc-ae9132076c33 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:47.577893 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46a23b-17c6-41d6-88fb-6ea9a12f33aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:47.578562 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46a23b-17c6-41d6-88fb-6ea9a12f33aa admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:47.578844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46a23b-17c6-41d6-88fb-6ea9a12f33aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:47.579115 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c46a23b-17c6-41d6-88fb-6ea9a12f33aa admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:47.579780 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46a23b-17c6-41d6-88fb-6ea9a12f33aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:47.582468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c46a23b-17c6-41d6-88fb-6ea9a12f33aa admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:47.599010 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6803a-8b4e-470b-a880-3ea801e057c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:47.599649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6803a-8b4e-470b-a880-3ea801e057c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:47.599938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6803a-8b4e-470b-a880-3ea801e057c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:47.600218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf6803a-8b4e-470b-a880-3ea801e057c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:47.600663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:47.600945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6803a-8b4e-470b-a880-3ea801e057c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:47.603407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf6803a-8b4e-470b-a880-3ea801e057c9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:47.610127 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 906/1814] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:28:47 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:28:47.615219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46a23b-17c6-41d6-88fb-6ea9a12f33aa admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:47.632395 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c46a23b-17c6-41d6-88fb-6ea9a12f33aa admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:47.638321 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24b038-ce31-46cd-9931-b4caa61b3ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:47.639015 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24b038-ce31-46cd-9931-b4caa61b3ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:47.639374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24b038-ce31-46cd-9931-b4caa61b3ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:47.639764 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24b038-ce31-46cd-9931-b4caa61b3ed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:47.640696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f24b038-ce31-46cd-9931-b4caa61b3ed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:47.642751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f24b038-ce31-46cd-9931-b4caa61b3ed9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:47.656740 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 909/1815] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:47 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:47.661262 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6803a-8b4e-470b-a880-3ea801e057c9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:47.668475 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:47.682604 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 907/1816] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:47.686691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24b038-ce31-46cd-9931-b4caa61b3ed9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:47.709847 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24b038-ce31-46cd-9931-b4caa61b3ed9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:47.813930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf6803a-8b4e-470b-a880-3ea801e057c9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:47.826937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa8b7f-4b2d-4409-a7a3-6d7faa743a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:47.827633 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa8b7f-4b2d-4409-a7a3-6d7faa743a56 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:47.827919 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa8b7f-4b2d-4409-a7a3-6d7faa743a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:47.828374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fa8b7f-4b2d-4409-a7a3-6d7faa743a56 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:47.829058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa8b7f-4b2d-4409-a7a3-6d7faa743a56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:47.831979 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fa8b7f-4b2d-4409-a7a3-6d7faa743a56 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:47.833411 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78145b-baf3-4660-bf5b-10c0fda5a065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:47.834495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78145b-baf3-4660-bf5b-10c0fda5a065 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:47.834945 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78145b-baf3-4660-bf5b-10c0fda5a065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:47.835369 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78145b-baf3-4660-bf5b-10c0fda5a065 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:47.836614 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78145b-baf3-4660-bf5b-10c0fda5a065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:47.838426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78145b-baf3-4660-bf5b-10c0fda5a065 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:47.852375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 908/1817] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:47.853808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78145b-baf3-4660-bf5b-10c0fda5a065 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:47.857224 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8d78145b-baf3-4660-bf5b-10c0fda5a065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:47.875272 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78145b-baf3-4660-bf5b-10c0fda5a065 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:47.883163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 910/1818] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:28:47 2022] POST /identity/v3/users => generated 292 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:47.883163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa8b7f-4b2d-4409-a7a3-6d7faa743a56 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:47.886774 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05194cef-8770-43de-b253-09821b980009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:47.887502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05194cef-8770-43de-b253-09821b980009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:47.887855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05194cef-8770-43de-b253-09821b980009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:47.888199 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05194cef-8770-43de-b253-09821b980009 admin admin] PATH_INFO: `/v3/projects/b3aaa3e46bba42d9a66c1ba8d355d4ec` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:47.889721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05194cef-8770-43de-b253-09821b980009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3aaa3e46bba42d9a66c1ba8d355d4ec)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:47.892724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05194cef-8770-43de-b253-09821b980009 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:47.897431 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fa8b7f-4b2d-4409-a7a3-6d7faa743a56 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:47.903313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16024a-5fe9-4b2e-a020-140658659d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:47.904060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16024a-5fe9-4b2e-a020-140658659d26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:47.904706 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16024a-5fe9-4b2e-a020-140658659d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:47.905003 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b16024a-5fe9-4b2e-a020-140658659d26 admin admin] PATH_INFO: `/v3/projects/6b7ce49305f64498aa9ae6093f637c7f/users/8848a94ad1814ff38d41931ec51925a0/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:47.937266 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16024a-5fe9-4b2e-a020-140658659d26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b7ce49305f64498aa9ae6093f637c7f, user_id=8848a94ad1814ff38d41931ec51925a0, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:47.940133 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b16024a-5fe9-4b2e-a020-140658659d26 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:47.968190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 911/1819] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:47 2022] PUT /identity/v3/projects/6b7ce49305f64498aa9ae6093f637c7f/users/8848a94ad1814ff38d41931ec51925a0/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:47.978226 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16024a-5fe9-4b2e-a020-140658659d26 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:47.999893 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-05194cef-8770-43de-b253-09821b980009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3aaa3e46bba42d9a66c1ba8d355d4ec', 'request_id': 'req-05194cef-8770-43de-b253-09821b980009'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:48.011328 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-05194cef-8770-43de-b253-09821b980009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3aaa3e46bba42d9a66c1ba8d355d4ec', 'request_id': 'req-05194cef-8770-43de-b253-09821b980009'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:48.022513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-05194cef-8770-43de-b253-09821b980009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3aaa3e46bba42d9a66c1ba8d355d4ec', 'request_id': 'req-05194cef-8770-43de-b253-09821b980009'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:48.050146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-05194cef-8770-43de-b253-09821b980009 admin admin] The token cache is being invalidate because project b3aaa3e46bba42d9a66c1ba8d355d4ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:28:48.050730 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-05194cef-8770-43de-b253-09821b980009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:48.053492 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-05194cef-8770-43de-b253-09821b980009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:48.056271 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 909/1820] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:28:47 2022] DELETE /identity/v3/projects/b3aaa3e46bba42d9a66c1ba8d355d4ec => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:48.064554 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de7bc3-9888-468a-acd2-ed646e70b7c5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:48.064907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de7bc3-9888-468a-acd2-ed646e70b7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:48.069599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46de7bc3-9888-468a-acd2-ed646e70b7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:48.097749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b16024a-5fe9-4b2e-a020-140658659d26 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:48.117993 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-46de7bc3-9888-468a-acd2-ed646e70b7c5 None None] MFA Rules not processed for user `8848a94ad1814ff38d41931ec51925a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:48.119532 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-46de7bc3-9888-468a-acd2-ed646e70b7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:48.199600 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508d489-abf3-47f3-81a0-b7c11f2c5dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:48.200757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508d489-abf3-47f3-81a0-b7c11f2c5dcc admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:48.200870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:48.200960 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508d489-abf3-47f3-81a0-b7c11f2c5dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:48.200960 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508d489-abf3-47f3-81a0-b7c11f2c5dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:48.201594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1508d489-abf3-47f3-81a0-b7c11f2c5dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:48.203242 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1508d489-abf3-47f3-81a0-b7c11f2c5dcc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:48.216203 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 910/1821] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:48.218387 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bbcb5-b01d-43be-afe8-819176e3978c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:48.218972 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bbcb5-b01d-43be-afe8-819176e3978c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:48.219453 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bbcb5-b01d-43be-afe8-819176e3978c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:48.232492 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 912/1822] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:48.233535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508d489-abf3-47f3-81a0-b7c11f2c5dcc admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:48.236839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1508d489-abf3-47f3-81a0-b7c11f2c5dcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:48.275764 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508d489-abf3-47f3-81a0-b7c11f2c5dcc admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:48.281365 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b2d16-0af6-4c8f-a8cf-1970a5cf9151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:48.282134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b2d16-0af6-4c8f-a8cf-1970a5cf9151 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:48.282493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b2d16-0af6-4c8f-a8cf-1970a5cf9151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:48.283255 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b2d16-0af6-4c8f-a8cf-1970a5cf9151 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:48.283584 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b2d16-0af6-4c8f-a8cf-1970a5cf9151 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:48.287042 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b2d16-0af6-4c8f-a8cf-1970a5cf9151 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:48.287332 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-978bbcb5-b01d-43be-afe8-819176e3978c None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:48.289872 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-978bbcb5-b01d-43be-afe8-819176e3978c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:48.303664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:48.314908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 911/1823] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:48.318704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bbcb5-b01d-43be-afe8-819176e3978c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:48.320739 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-978bbcb5-b01d-43be-afe8-819176e3978c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:48.355514 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bbcb5-b01d-43be-afe8-819176e3978c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:48.359209 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 913/1824] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:48 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:48.360771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70456486-350d-4971-ab69-d2deee572d2a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:48.361071 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70456486-350d-4971-ab69-d2deee572d2a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:48.361354 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70456486-350d-4971-ab69-d2deee572d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:48.403662 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-70456486-350d-4971-ab69-d2deee572d2a None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:48.405165 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-70456486-350d-4971-ab69-d2deee572d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:48.437968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148feb0-fa16-4e96-b51a-ac204a6c29e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:48.439292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148feb0-fa16-4e96-b51a-ac204a6c29e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:48.439725 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148feb0-fa16-4e96-b51a-ac204a6c29e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:48.440660 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148feb0-fa16-4e96-b51a-ac204a6c29e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:48.441602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148feb0-fa16-4e96-b51a-ac204a6c29e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:48.443723 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f148feb0-fa16-4e96-b51a-ac204a6c29e3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:48.456927 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 912/1825] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:48.458063 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148feb0-fa16-4e96-b51a-ac204a6c29e3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:48.472943 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f148feb0-fa16-4e96-b51a-ac204a6c29e3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:48.479889 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e27ed3-af40-4eb6-a5d0-811ac04a55a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:48.481344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e27ed3-af40-4eb6-a5d0-811ac04a55a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:48.481809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e27ed3-af40-4eb6-a5d0-811ac04a55a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:48.484988 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e27ed3-af40-4eb6-a5d0-811ac04a55a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:48.484988 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e27ed3-af40-4eb6-a5d0-811ac04a55a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:48.486163 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e27ed3-af40-4eb6-a5d0-811ac04a55a4 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:48.492300 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 914/1826] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:48.493199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70456486-350d-4971-ab69-d2deee572d2a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:48.507742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70456486-350d-4971-ab69-d2deee572d2a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:48.522385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 913/1827] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:28:48 2022] POST /identity/v3/users => generated 293 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:48.523742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e27ed3-af40-4eb6-a5d0-811ac04a55a4 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:48.524094 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d4cb9-aaa3-42b6-a3ad-4ecaef9781aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:48.524844 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d4cb9-aaa3-42b6-a3ad-4ecaef9781aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:48.525161 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d4cb9-aaa3-42b6-a3ad-4ecaef9781aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:48.525580 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67d4cb9-aaa3-42b6-a3ad-4ecaef9781aa admin admin] PATH_INFO: `/v3/projects/8b9c708b9ee44a3b9d7d49737a07c4ba/users/b5cea747a00e4f09af2eec28663bb335/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:48.537471 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e27ed3-af40-4eb6-a5d0-811ac04a55a4 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:48.543969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954899f0-9578-447c-893b-47e77f6af699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:48.544594 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954899f0-9578-447c-893b-47e77f6af699 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:48.544881 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954899f0-9578-447c-893b-47e77f6af699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:48.545149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954899f0-9578-447c-893b-47e77f6af699 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:48.546248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954899f0-9578-447c-893b-47e77f6af699 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:28:48.547856 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d4cb9-aaa3-42b6-a3ad-4ecaef9781aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b9c708b9ee44a3b9d7d49737a07c4ba, user_id=b5cea747a00e4f09af2eec28663bb335, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:48.548337 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954899f0-9578-447c-893b-47e77f6af699 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:48.552231 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954899f0-9578-447c-893b-47e77f6af699 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:48.552528 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67d4cb9-aaa3-42b6-a3ad-4ecaef9781aa admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:48.578395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 915/1828] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:48 2022] PUT /identity/v3/projects/8b9c708b9ee44a3b9d7d49737a07c4ba/users/b5cea747a00e4f09af2eec28663bb335/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:48.582426 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d4cb9-aaa3-42b6-a3ad-4ecaef9781aa admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:48.583740 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 914/1829] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:28:48.592126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954899f0-9578-447c-893b-47e77f6af699 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:48.604625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954899f0-9578-447c-893b-47e77f6af699 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:48.687031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67d4cb9-aaa3-42b6-a3ad-4ecaef9781aa admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:48.696848 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36065f94-a8bd-4714-9894-6d33db8698a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:48.698072 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36065f94-a8bd-4714-9894-6d33db8698a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:48.698370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36065f94-a8bd-4714-9894-6d33db8698a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:48.698754 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36065f94-a8bd-4714-9894-6d33db8698a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:48.701572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36065f94-a8bd-4714-9894-6d33db8698a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:48.709716 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36065f94-a8bd-4714-9894-6d33db8698a4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:48.720609 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 916/1830] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:48.722049 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c48c5-a14a-4317-ae84-5bc9e8de22a5 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:48.722476 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c48c5-a14a-4317-ae84-5bc9e8de22a5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:48.722767 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c48c5-a14a-4317-ae84-5bc9e8de22a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:48.755225 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16459cff-48c8-4c3c-aac8-6946d43f3f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:48.756139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d42c48c5-a14a-4317-ae84-5bc9e8de22a5 None None] MFA Rules not processed for user `b5cea747a00e4f09af2eec28663bb335`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:48.756685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16459cff-48c8-4c3c-aac8-6946d43f3f45 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:48.757222 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16459cff-48c8-4c3c-aac8-6946d43f3f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:48.758006 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16459cff-48c8-4c3c-aac8-6946d43f3f45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:48.758214 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d42c48c5-a14a-4317-ae84-5bc9e8de22a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:48.759666 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16459cff-48c8-4c3c-aac8-6946d43f3f45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:48.762254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16459cff-48c8-4c3c-aac8-6946d43f3f45 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:48.801213 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 915/1831] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:48 2022] POST /identity/v3/projects => generated 393 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:48.802239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16459cff-48c8-4c3c-aac8-6946d43f3f45 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:48.835115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:48.852195 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 917/1832] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:48.853258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c48c5-a14a-4317-ae84-5bc9e8de22a5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:48.872297 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c48c5-a14a-4317-ae84-5bc9e8de22a5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:48.873109 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:48.899708 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16459cff-48c8-4c3c-aac8-6946d43f3f45 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:48.905011 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71ae24-0ab8-4a0f-bb41-f32d2a87bdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:48.905743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71ae24-0ab8-4a0f-bb41-f32d2a87bdb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:48.906131 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71ae24-0ab8-4a0f-bb41-f32d2a87bdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:48.906415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71ae24-0ab8-4a0f-bb41-f32d2a87bdb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:48.907332 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71ae24-0ab8-4a0f-bb41-f32d2a87bdb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:48.911690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71ae24-0ab8-4a0f-bb41-f32d2a87bdb9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:48.929734 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 916/1833] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:48.929734 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71ae24-0ab8-4a0f-bb41-f32d2a87bdb9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:48.942289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71ae24-0ab8-4a0f-bb41-f32d2a87bdb9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:48.948069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bcfe97-3fe8-43b8-9e3f-c294eb9a62ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:48.948670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcfe97-3fe8-43b8-9e3f-c294eb9a62ea admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:48.948956 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcfe97-3fe8-43b8-9e3f-c294eb9a62ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:48.949235 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bcfe97-3fe8-43b8-9e3f-c294eb9a62ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:48.949945 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bcfe97-3fe8-43b8-9e3f-c294eb9a62ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:48.952019 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bcfe97-3fe8-43b8-9e3f-c294eb9a62ea admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:48.962034 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783008d-2c8c-4e62-b5b5-bf928fdbf6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:48.963421 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783008d-2c8c-4e62-b5b5-bf928fdbf6c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:48.963879 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783008d-2c8c-4e62-b5b5-bf928fdbf6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:48.964304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783008d-2c8c-4e62-b5b5-bf928fdbf6c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:48.965138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783008d-2c8c-4e62-b5b5-bf928fdbf6c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:48.967293 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783008d-2c8c-4e62-b5b5-bf928fdbf6c1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:49.001067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 918/1834] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:28:48 2022] POST /identity/v3/users => generated 297 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:49.002607 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783008d-2c8c-4e62-b5b5-bf928fdbf6c1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:49.015628 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783008d-2c8c-4e62-b5b5-bf928fdbf6c1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:49.017945 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 917/1835] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:48 2022] POST /identity/v3/projects => generated 383 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:49.018905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bcfe97-3fe8-43b8-9e3f-c294eb9a62ea admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:49.095627 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b9db2c-5f35-4ba7-8638-7c8691f23924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:49.097156 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9db2c-5f35-4ba7-8638-7c8691f23924 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:49.097514 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9db2c-5f35-4ba7-8638-7c8691f23924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:49.098421 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b9db2c-5f35-4ba7-8638-7c8691f23924 admin admin] PATH_INFO: `/v3/projects/dbddbf13cfa74f609c7c06232e6f6524/users/67dc008ae157470e8b9e23c4c865493d/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:49.106710 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bcfe97-3fe8-43b8-9e3f-c294eb9a62ea admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:49.114369 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de64bfa-ccad-431c-991c-920d03459e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:49.115273 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de64bfa-ccad-431c-991c-920d03459e45 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:49.115740 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de64bfa-ccad-431c-991c-920d03459e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:49.116039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b9db2c-5f35-4ba7-8638-7c8691f23924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbddbf13cfa74f609c7c06232e6f6524, user_id=67dc008ae157470e8b9e23c4c865493d, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:49.116159 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de64bfa-ccad-431c-991c-920d03459e45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:49.116907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de64bfa-ccad-431c-991c-920d03459e45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:49.119495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de64bfa-ccad-431c-991c-920d03459e45 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:49.120555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b9db2c-5f35-4ba7-8638-7c8691f23924 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:49.143603 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 918/1837] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:49.150259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 919/1837] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:49 2022] PUT /identity/v3/projects/dbddbf13cfa74f609c7c06232e6f6524/users/67dc008ae157470e8b9e23c4c865493d/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:49.152348 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77261a31-15ca-4f9a-aa68-751840ce8e93 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:49.152647 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77261a31-15ca-4f9a-aa68-751840ce8e93 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:49.153014 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77261a31-15ca-4f9a-aa68-751840ce8e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:49.154391 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de64bfa-ccad-431c-991c-920d03459e45 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:49.167496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de64bfa-ccad-431c-991c-920d03459e45 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:49.192584 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-77261a31-15ca-4f9a-aa68-751840ce8e93 None None] MFA Rules not processed for user `67dc008ae157470e8b9e23c4c865493d`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:49.193846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-77261a31-15ca-4f9a-aa68-751840ce8e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:49.231739 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179152c1-8696-43c7-9633-3bc692b5a9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:49.232285 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179152c1-8696-43c7-9633-3bc692b5a9d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:49.232567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179152c1-8696-43c7-9633-3bc692b5a9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:49.232850 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179152c1-8696-43c7-9633-3bc692b5a9d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:49.233542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179152c1-8696-43c7-9633-3bc692b5a9d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:49.235541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179152c1-8696-43c7-9633-3bc692b5a9d6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:49.262384 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 919/1838] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:28:49 2022] POST /identity/v3/users => generated 298 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:49.264396 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179152c1-8696-43c7-9633-3bc692b5a9d6 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:49.278030 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:49.295642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 920/1839] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:49.296523 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77261a31-15ca-4f9a-aa68-751840ce8e93 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:49.301902 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179152c1-8696-43c7-9633-3bc692b5a9d6 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:49.310685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077a8b1-b7d2-4fbb-88c7-0607fea4f352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:49.310998 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77261a31-15ca-4f9a-aa68-751840ce8e93 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:49.311861 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077a8b1-b7d2-4fbb-88c7-0607fea4f352 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:49.312411 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077a8b1-b7d2-4fbb-88c7-0607fea4f352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:49.313093 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7077a8b1-b7d2-4fbb-88c7-0607fea4f352 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:49.314227 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7077a8b1-b7d2-4fbb-88c7-0607fea4f352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:49.316216 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7077a8b1-b7d2-4fbb-88c7-0607fea4f352 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:49.316746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09950ef9-b38b-45fa-b66f-16a7a7b553b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:49.317388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09950ef9-b38b-45fa-b66f-16a7a7b553b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:49.317854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09950ef9-b38b-45fa-b66f-16a7a7b553b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:49.318137 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09950ef9-b38b-45fa-b66f-16a7a7b553b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:49.318853 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09950ef9-b38b-45fa-b66f-16a7a7b553b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:49.320625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09950ef9-b38b-45fa-b66f-16a7a7b553b9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:49.338705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 920/1840] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:49.339718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077a8b1-b7d2-4fbb-88c7-0607fea4f352 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:49.340994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 921/1841] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:49.341936 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09950ef9-b38b-45fa-b66f-16a7a7b553b9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:49.353446 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7077a8b1-b7d2-4fbb-88c7-0607fea4f352 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:49.359874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a75656-a232-4def-ac63-d37aaf32aa49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:49.360312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09950ef9-b38b-45fa-b66f-16a7a7b553b9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:49.360691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a75656-a232-4def-ac63-d37aaf32aa49 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:49.360995 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a75656-a232-4def-ac63-d37aaf32aa49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:49.361273 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a75656-a232-4def-ac63-d37aaf32aa49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:49.362254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a75656-a232-4def-ac63-d37aaf32aa49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:49.364529 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a75656-a232-4def-ac63-d37aaf32aa49 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:49.380954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c73b8c-2448-4303-87d7-b0be2bb74312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:49.381906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c73b8c-2448-4303-87d7-b0be2bb74312 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:49.382387 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c73b8c-2448-4303-87d7-b0be2bb74312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:49.383176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c73b8c-2448-4303-87d7-b0be2bb74312 admin admin] PATH_INFO: `/v3/projects/056baa152e8e4753afd98ca02525e9cc/users/d9d9656e782d4458ac4cc873c912ee92/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:49.410868 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 921/1842] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:49 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:49.415056 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a75656-a232-4def-ac63-d37aaf32aa49 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:49.421626 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c73b8c-2448-4303-87d7-b0be2bb74312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=056baa152e8e4753afd98ca02525e9cc, user_id=d9d9656e782d4458ac4cc873c912ee92, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:49.424617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c73b8c-2448-4303-87d7-b0be2bb74312 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:49.434871 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:49.451726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 922/1843] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:49 2022] PUT /identity/v3/projects/056baa152e8e4753afd98ca02525e9cc/users/d9d9656e782d4458ac4cc873c912ee92/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:49.453666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c73b8c-2448-4303-87d7-b0be2bb74312 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:49.525705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a75656-a232-4def-ac63-d37aaf32aa49 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:49.568718 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c73b8c-2448-4303-87d7-b0be2bb74312 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:49.575348 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432c8995-323a-430a-b225-7a110ad1d5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:49.576170 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c8995-323a-430a-b225-7a110ad1d5c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:49.576459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c8995-323a-430a-b225-7a110ad1d5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:49.576744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432c8995-323a-430a-b225-7a110ad1d5c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:49.577882 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c8995-323a-430a-b225-7a110ad1d5c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:49.580015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432c8995-323a-430a-b225-7a110ad1d5c9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:49.614404 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 923/1844] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:28:49 2022] POST /identity/v3/users => generated 298 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:49.615998 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717680b-a2eb-4593-890e-91fb64028a24 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:49.616289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717680b-a2eb-4593-890e-91fb64028a24 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:49.616713 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717680b-a2eb-4593-890e-91fb64028a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:49.650294 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d6eb7-bd2f-400c-91cc-a4139390e8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:49.651039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d6eb7-bd2f-400c-91cc-a4139390e8a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:49.651259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d6eb7-bd2f-400c-91cc-a4139390e8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:49.652475 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d6eb7-bd2f-400c-91cc-a4139390e8a7 admin admin] PATH_INFO: `/v3/projects/e77b4433bf344605885778d659f601e0/users/9b36c300e6424829843c56eaae049234/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:49.658140 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c717680b-a2eb-4593-890e-91fb64028a24 None None] MFA Rules not processed for user `d9d9656e782d4458ac4cc873c912ee92`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:49.660629 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c717680b-a2eb-4593-890e-91fb64028a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:49.673773 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d6eb7-bd2f-400c-91cc-a4139390e8a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e77b4433bf344605885778d659f601e0, user_id=9b36c300e6424829843c56eaae049234, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:49.679004 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d6eb7-bd2f-400c-91cc-a4139390e8a7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:49.719308 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 922/1845] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:49 2022] PUT /identity/v3/projects/e77b4433bf344605885778d659f601e0/users/9b36c300e6424829843c56eaae049234/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:49.721232 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d6eb7-bd2f-400c-91cc-a4139390e8a7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:49.790359 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 924/1846] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:49.792293 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717680b-a2eb-4593-890e-91fb64028a24 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:49.810747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d6eb7-bd2f-400c-91cc-a4139390e8a7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:49.814219 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717680b-a2eb-4593-890e-91fb64028a24 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:49.816411 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1b8a6-c0d1-40d3-af76-afff97e00f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:49.817782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1b8a6-c0d1-40d3-af76-afff97e00f34 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:49.818127 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1b8a6-c0d1-40d3-af76-afff97e00f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:49.818412 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e1b8a6-c0d1-40d3-af76-afff97e00f34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:49.819134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1b8a6-c0d1-40d3-af76-afff97e00f34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:49.821718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e1b8a6-c0d1-40d3-af76-afff97e00f34 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:49.823917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7142a-1895-42d3-9ed1-132955e67719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:49.825235 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7142a-1895-42d3-9ed1-132955e67719 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:49.825705 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7142a-1895-42d3-9ed1-132955e67719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:49.826125 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e7142a-1895-42d3-9ed1-132955e67719 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:49.828124 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7142a-1895-42d3-9ed1-132955e67719 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:49.832373 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e7142a-1895-42d3-9ed1-132955e67719 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:49.840690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 923/1847] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:49.840690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1b8a6-c0d1-40d3-af76-afff97e00f34 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:49.853003 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e1b8a6-c0d1-40d3-af76-afff97e00f34 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:49.860821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874fd6da-1b28-46d4-9b94-916fae86c429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:49.862448 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874fd6da-1b28-46d4-9b94-916fae86c429 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:49.863133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874fd6da-1b28-46d4-9b94-916fae86c429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:49.863407 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874fd6da-1b28-46d4-9b94-916fae86c429 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:49.865046 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 925/1848] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:49 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:49.866730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7142a-1895-42d3-9ed1-132955e67719 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:49.868748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874fd6da-1b28-46d4-9b94-916fae86c429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:49.881406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874fd6da-1b28-46d4-9b94-916fae86c429 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:49.883087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e7142a-1895-42d3-9ed1-132955e67719 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:49.889861 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1a9b8-068e-47d2-a213-904d71a051a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:49.890759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1a9b8-068e-47d2-a213-904d71a051a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:49.891203 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1a9b8-068e-47d2-a213-904d71a051a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:49.891644 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1a9b8-068e-47d2-a213-904d71a051a6 admin admin] PATH_INFO: `/v3/projects/9374c631c31442c482ffae6c6ca5af7e/users/f12ba0b8d7a24aa7bc046e30987df4e0/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:49.921604 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1a9b8-068e-47d2-a213-904d71a051a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9374c631c31442c482ffae6c6ca5af7e, user_id=f12ba0b8d7a24aa7bc046e30987df4e0, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:49.924274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 924/1849] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:49 2022] POST /identity/v3/projects => generated 393 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:49.926027 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874fd6da-1b28-46d4-9b94-916fae86c429 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:49.928070 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1a9b8-068e-47d2-a213-904d71a051a6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:49.946241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:49.954998 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 926/1850] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:49 2022] PUT /identity/v3/projects/9374c631c31442c482ffae6c6ca5af7e/users/f12ba0b8d7a24aa7bc046e30987df4e0/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:49.958162 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1a9b8-068e-47d2-a213-904d71a051a6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:49.966119 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:50.028032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874fd6da-1b28-46d4-9b94-916fae86c429 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:50.154546 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1a9b8-068e-47d2-a213-904d71a051a6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:50.168703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf089ee-b21b-4bf0-963f-f5b56ae61f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:50.169399 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf089ee-b21b-4bf0-963f-f5b56ae61f55 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:50.169764 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf089ee-b21b-4bf0-963f-f5b56ae61f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:50.170118 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf089ee-b21b-4bf0-963f-f5b56ae61f55 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:50.171029 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf089ee-b21b-4bf0-963f-f5b56ae61f55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:50.173080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf089ee-b21b-4bf0-963f-f5b56ae61f55 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:50.201464 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 927/1851] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:28:49 2022] POST /identity/v3/users => generated 303 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:50.202349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf089ee-b21b-4bf0-963f-f5b56ae61f55 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:50.207895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66dce0-e86d-4b03-a3d9-b6746860217a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:50.208936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66dce0-e86d-4b03-a3d9-b6746860217a admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:50.209281 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66dce0-e86d-4b03-a3d9-b6746860217a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:50.209651 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da66dce0-e86d-4b03-a3d9-b6746860217a admin admin] PATH_INFO: `/v3/projects/e77b4433bf344605885778d659f601e0/users/9b36c300e6424829843c56eaae049234/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:50.212225 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da66dce0-e86d-4b03-a3d9-b6746860217a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e77b4433bf344605885778d659f601e0, user_id=9b36c300e6424829843c56eaae049234, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:50.215047 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf089ee-b21b-4bf0-963f-f5b56ae61f55 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:50.215502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da66dce0-e86d-4b03-a3d9-b6746860217a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:50.220535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bce3f-74bd-4029-ab2b-635421246de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:50.221165 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bce3f-74bd-4029-ab2b-635421246de8 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:50.222170 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bce3f-74bd-4029-ab2b-635421246de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:50.222486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45bce3f-74bd-4029-ab2b-635421246de8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:50.223353 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45bce3f-74bd-4029-ab2b-635421246de8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:50.225076 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45bce3f-74bd-4029-ab2b-635421246de8 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:50.233387 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 925/1852] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:49 2022] PUT /identity/v3/projects/e77b4433bf344605885778d659f601e0/users/9b36c300e6424829843c56eaae049234/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:50.234639 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66dce0-e86d-4b03-a3d9-b6746860217a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:50.237719 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 928/1853] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:50.241356 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bce3f-74bd-4029-ab2b-635421246de8 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:50.255242 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45bce3f-74bd-4029-ab2b-635421246de8 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:50.311317 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da66dce0-e86d-4b03-a3d9-b6746860217a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:50.317487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349bb7fb-c8a8-4d97-b6c4-64b7ed6e54b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:50.318218 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bb7fb-c8a8-4d97-b6c4-64b7ed6e54b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:50.318567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bb7fb-c8a8-4d97-b6c4-64b7ed6e54b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:50.318929 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bb7fb-c8a8-4d97-b6c4-64b7ed6e54b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:50.319751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349bb7fb-c8a8-4d97-b6c4-64b7ed6e54b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:50.321418 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349bb7fb-c8a8-4d97-b6c4-64b7ed6e54b1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:50.337887 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9dc43-76d6-468c-84a0-92f2cea2bce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:50.338574 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9dc43-76d6-468c-84a0-92f2cea2bce0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:50.338862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9dc43-76d6-468c-84a0-92f2cea2bce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:50.339137 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb9dc43-76d6-468c-84a0-92f2cea2bce0 admin admin] PATH_INFO: `/v3/projects/9374c631c31442c482ffae6c6ca5af7e/users/f12ba0b8d7a24aa7bc046e30987df4e0/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:50.340391 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 926/1854] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:50.341839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551ee41-dbe4-4005-8cb0-70f2886b5038 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:50.342118 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551ee41-dbe4-4005-8cb0-70f2886b5038 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:50.342392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0551ee41-dbe4-4005-8cb0-70f2886b5038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:50.346513 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9dc43-76d6-468c-84a0-92f2cea2bce0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9374c631c31442c482ffae6c6ca5af7e, user_id=f12ba0b8d7a24aa7bc046e30987df4e0, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:50.349273 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb9dc43-76d6-468c-84a0-92f2cea2bce0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:50.370109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 929/1855] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:50 2022] PUT /identity/v3/projects/9374c631c31442c482ffae6c6ca5af7e/users/f12ba0b8d7a24aa7bc046e30987df4e0/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:28:50.370109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9dc43-76d6-468c-84a0-92f2cea2bce0 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:50.390936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0551ee41-dbe4-4005-8cb0-70f2886b5038 None None] MFA Rules not processed for user `9b36c300e6424829843c56eaae049234`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:50.392212 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0551ee41-dbe4-4005-8cb0-70f2886b5038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:50.435291 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:50.464947 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb9dc43-76d6-468c-84a0-92f2cea2bce0 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:50.473127 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cfc267-7a5b-4a90-ac15-cfad2c083607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:50.474369 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cfc267-7a5b-4a90-ac15-cfad2c083607 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:50.474870 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cfc267-7a5b-4a90-ac15-cfad2c083607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:50.475357 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cfc267-7a5b-4a90-ac15-cfad2c083607 admin admin] PATH_INFO: `/v3/projects/c6f6bd5299e24051be854e1361b93fea/users/80b7089f983f4aac97b2e98cc7065008/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:50.496512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 927/1856] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:50.498217 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cfc267-7a5b-4a90-ac15-cfad2c083607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6f6bd5299e24051be854e1361b93fea, user_id=80b7089f983f4aac97b2e98cc7065008, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:50.498618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dbfc8-1048-4d0e-ab78-7dec7cdd91df None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:50.498980 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dbfc8-1048-4d0e-ab78-7dec7cdd91df None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:50.499309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761dbfc8-1048-4d0e-ab78-7dec7cdd91df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:50.512177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cfc267-7a5b-4a90-ac15-cfad2c083607 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:50.529203 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:50.536632 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 930/1857] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:50 2022] PUT /identity/v3/projects/c6f6bd5299e24051be854e1361b93fea/users/80b7089f983f4aac97b2e98cc7065008/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:50.538012 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cfc267-7a5b-4a90-ac15-cfad2c083607 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:50.541611 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-761dbfc8-1048-4d0e-ab78-7dec7cdd91df None None] MFA Rules not processed for user `f12ba0b8d7a24aa7bc046e30987df4e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:50.542968 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d4cfc267-7a5b-4a90-ac15-cfad2c083607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:50.543540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-761dbfc8-1048-4d0e-ab78-7dec7cdd91df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:50.622364 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cfc267-7a5b-4a90-ac15-cfad2c083607 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:50.627410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362d9b8-091c-4735-a8f7-1d407a5f86bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:50.628783 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362d9b8-091c-4735-a8f7-1d407a5f86bf service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:50.629904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362d9b8-091c-4735-a8f7-1d407a5f86bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:50.629904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362d9b8-091c-4735-a8f7-1d407a5f86bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:50.688700 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 928/1858] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:50.690650 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761dbfc8-1048-4d0e-ab78-7dec7cdd91df None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:50.703277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362d9b8-091c-4735-a8f7-1d407a5f86bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:50.705051 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3362d9b8-091c-4735-a8f7-1d407a5f86bf service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:50.731262 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 931/1859] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:50.733027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362d9b8-091c-4735-a8f7-1d407a5f86bf service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:50.738414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3362d9b8-091c-4735-a8f7-1d407a5f86bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:50.760739 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761dbfc8-1048-4d0e-ab78-7dec7cdd91df None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:50.766619 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fac0d5-1bd8-476e-ae14-180f6a3e335b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:50.767240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fac0d5-1bd8-476e-ae14-180f6a3e335b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:50.767521 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fac0d5-1bd8-476e-ae14-180f6a3e335b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:50.767789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fac0d5-1bd8-476e-ae14-180f6a3e335b admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:50.768813 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362d9b8-091c-4735-a8f7-1d407a5f86bf service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:50.769014 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fac0d5-1bd8-476e-ae14-180f6a3e335b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:50.770735 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fac0d5-1bd8-476e-ae14-180f6a3e335b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:50.775316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d832da-76a6-4fe4-8427-7f2c9e6bfcc0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:50.776696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d832da-76a6-4fe4-8427-7f2c9e6bfcc0 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:50.777491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d832da-76a6-4fe4-8427-7f2c9e6bfcc0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:50.778093 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d832da-76a6-4fe4-8427-7f2c9e6bfcc0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:50.781806 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 929/1860] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:50 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:50.786111 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fac0d5-1bd8-476e-ae14-180f6a3e335b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:50.797211 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fac0d5-1bd8-476e-ae14-180f6a3e335b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:50.803208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375771f0-04f6-4df9-b32e-51e4d294fcc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:50.804524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375771f0-04f6-4df9-b32e-51e4d294fcc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:50.804879 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375771f0-04f6-4df9-b32e-51e4d294fcc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:50.805194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375771f0-04f6-4df9-b32e-51e4d294fcc2 admin admin] PATH_INFO: `/v3/projects/c6f6bd5299e24051be854e1361b93fea/users/80b7089f983f4aac97b2e98cc7065008/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:50.808075 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375771f0-04f6-4df9-b32e-51e4d294fcc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6f6bd5299e24051be854e1361b93fea, user_id=80b7089f983f4aac97b2e98cc7065008, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:50.810844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375771f0-04f6-4df9-b32e-51e4d294fcc2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:50.825450 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 930/1861] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:50 2022] PUT /identity/v3/projects/c6f6bd5299e24051be854e1361b93fea/users/80b7089f983f4aac97b2e98cc7065008/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:50.835778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605e3fd-90a9-4c50-9793-447d58b18cbb None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:50.836147 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605e3fd-90a9-4c50-9793-447d58b18cbb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:50.836429 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605e3fd-90a9-4c50-9793-447d58b18cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:50.849409 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d832da-76a6-4fe4-8427-7f2c9e6bfcc0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:50.851436 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d832da-76a6-4fe4-8427-7f2c9e6bfcc0 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:50.865933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 932/1862] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:50.868471 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8605e3fd-90a9-4c50-9793-447d58b18cbb None None] MFA Rules not processed for user `80b7089f983f4aac97b2e98cc7065008`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:50.869017 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8605e3fd-90a9-4c50-9793-447d58b18cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:50.901952 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:50.954825 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 931/1863] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:28:51.074543 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d832da-76a6-4fe4-8427-7f2c9e6bfcc0 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:51.083098 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d832da-76a6-4fe4-8427-7f2c9e6bfcc0 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:51.173958 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d17f89-57e6-4bc2-9a22-000c62b29419 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:51.174658 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d17f89-57e6-4bc2-9a22-000c62b29419 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:51.175060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d17f89-57e6-4bc2-9a22-000c62b29419 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:51.175674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d17f89-57e6-4bc2-9a22-000c62b29419 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:51.187406 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:51.192753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d17f89-57e6-4bc2-9a22-000c62b29419 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:51.195282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d17f89-57e6-4bc2-9a22-000c62b29419 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:51.222308 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 933/1864] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:51 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:51.270596 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605e3fd-90a9-4c50-9793-447d58b18cbb None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:51.289361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605e3fd-90a9-4c50-9793-447d58b18cbb None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:51.294519 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4435a-5145-43bf-befa-18a98ec5e05c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:51.297021 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4435a-5145-43bf-befa-18a98ec5e05c service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:51.297399 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4435a-5145-43bf-befa-18a98ec5e05c service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:51.298338 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4435a-5145-43bf-befa-18a98ec5e05c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:51.424553 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4435a-5145-43bf-befa-18a98ec5e05c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:51.427034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd4435a-5145-43bf-befa-18a98ec5e05c service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:51.439928 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 932/1865] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:51.741163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e1de9-b81b-45bd-ad32-af5889a117b8 None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:51.741613 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e1de9-b81b-45bd-ad32-af5889a117b8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:51.741895 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e1de9-b81b-45bd-ad32-af5889a117b8 None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:51.745601 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 934/1866] 10.222.0.39 () {54 vars in 921 bytes} [Thu Jul 28 22:28:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:28:51.751035 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef26f67-eab7-4d35-98dc-c85982f96e5d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:51.751911 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef26f67-eab7-4d35-98dc-c85982f96e5d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:51.753410 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef26f67-eab7-4d35-98dc-c85982f96e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:51.787150 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-fef26f67-eab7-4d35-98dc-c85982f96e5d None None] MFA Rules not processed for user `77efe8046ea0468cb0c7c58c24fc4e75`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:51.788395 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fef26f67-eab7-4d35-98dc-c85982f96e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:51.895467 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 933/1867] 10.222.0.39 () {58 vars in 1018 bytes} [Thu Jul 28 22:28:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:51.909403 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e1de9-b81b-45bd-ad32-af5889a117b8 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:51.911453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-355e1de9-b81b-45bd-ad32-af5889a117b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:51.930285 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e1de9-b81b-45bd-ad32-af5889a117b8 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:51.936302 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150e010-201d-4129-8cd2-f181341cfc83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:51.936938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150e010-201d-4129-8cd2-f181341cfc83 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:51.937306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150e010-201d-4129-8cd2-f181341cfc83 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:51.937617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150e010-201d-4129-8cd2-f181341cfc83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:51.947497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150e010-201d-4129-8cd2-f181341cfc83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:51.948868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150e010-201d-4129-8cd2-f181341cfc83 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:51.973270 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 935/1868] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:52.713470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef26f67-eab7-4d35-98dc-c85982f96e5d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:52.723804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef26f67-eab7-4d35-98dc-c85982f96e5d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:52.728042 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3baa6-c98d-41cd-b281-f878608bcb27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:52.729078 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3baa6-c98d-41cd-b281-f878608bcb27 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:52.729511 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3baa6-c98d-41cd-b281-f878608bcb27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:52.729967 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d3baa6-c98d-41cd-b281-f878608bcb27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:52.845384 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:52.852137 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3baa6-c98d-41cd-b281-f878608bcb27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:52.855523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d3baa6-c98d-41cd-b281-f878608bcb27 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:52.877503 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 934/1869] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:53.568522 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150e010-201d-4129-8cd2-f181341cfc83 service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:53.580666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150e010-201d-4129-8cd2-f181341cfc83 service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:53.585949 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360af15f-18ef-4af8-842b-82c1c40d9650 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:53.587401 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360af15f-18ef-4af8-842b-82c1c40d9650 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:53.587777 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360af15f-18ef-4af8-842b-82c1c40d9650 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:53.588057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360af15f-18ef-4af8-842b-82c1c40d9650 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:53.660441 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360af15f-18ef-4af8-842b-82c1c40d9650 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:53.662987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360af15f-18ef-4af8-842b-82c1c40d9650 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:53.679059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 936/1870] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:55.340416 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3baa6-c98d-41cd-b281-f878608bcb27 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:55.449374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d3baa6-c98d-41cd-b281-f878608bcb27 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:55.454907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d37191-4858-47b9-a69a-51d6e9d091e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:55.455781 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d37191-4858-47b9-a69a-51d6e9d091e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:55.456127 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d37191-4858-47b9-a69a-51d6e9d091e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:55.456469 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d37191-4858-47b9-a69a-51d6e9d091e9 admin admin] PATH_INFO: `/v3/users/d9d9656e782d4458ac4cc873c912ee92` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:55.457752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d37191-4858-47b9-a69a-51d6e9d091e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9d9656e782d4458ac4cc873c912ee92)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:55.459967 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d37191-4858-47b9-a69a-51d6e9d091e9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:55.510247 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d5d37191-4858-47b9-a69a-51d6e9d091e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9d9656e782d4458ac4cc873c912ee92'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:55.517432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d5d37191-4858-47b9-a69a-51d6e9d091e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9d9656e782d4458ac4cc873c912ee92'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:55.518871 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d5d37191-4858-47b9-a69a-51d6e9d091e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9d9656e782d4458ac4cc873c912ee92'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:55.519499 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d5d37191-4858-47b9-a69a-51d6e9d091e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9d9656e782d4458ac4cc873c912ee92'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:55.531583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d5d37191-4858-47b9-a69a-51d6e9d091e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9d9656e782d4458ac4cc873c912ee92'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:55.546933 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 935/1871] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:28:55 2022] DELETE /identity/v3/users/d9d9656e782d4458ac4cc873c912ee92 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:55.551804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66fe5b-830a-405d-94f2-c68fbb1a7d8f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:55.552263 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66fe5b-830a-405d-94f2-c68fbb1a7d8f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:55.552373 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b66fe5b-830a-405d-94f2-c68fbb1a7d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:55.572292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1b66fe5b-830a-405d-94f2-c68fbb1a7d8f None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:55.573861 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1b66fe5b-830a-405d-94f2-c68fbb1a7d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:55.709881 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 937/1872] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:55.711263 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d37191-4858-47b9-a69a-51d6e9d091e9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:55.712890 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d5d37191-4858-47b9-a69a-51d6e9d091e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:55.753114 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66fe5b-830a-405d-94f2-c68fbb1a7d8f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:55.754717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1b66fe5b-830a-405d-94f2-c68fbb1a7d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:55.811586 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d37191-4858-47b9-a69a-51d6e9d091e9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:55.817342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f203b-a28d-45eb-970d-e02764ae258a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:55.818259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f203b-a28d-45eb-970d-e02764ae258a service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:55.818636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f203b-a28d-45eb-970d-e02764ae258a service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:55.819172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f203b-a28d-45eb-970d-e02764ae258a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:55.831563 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b66fe5b-830a-405d-94f2-c68fbb1a7d8f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:55.833229 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f203b-a28d-45eb-970d-e02764ae258a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:55.835139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007f203b-a28d-45eb-970d-e02764ae258a service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:55.839701 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739cf93c-87a4-454a-9ab2-86290a976334 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:55.840340 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739cf93c-87a4-454a-9ab2-86290a976334 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:55.840786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739cf93c-87a4-454a-9ab2-86290a976334 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:55.841213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739cf93c-87a4-454a-9ab2-86290a976334 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:55.843371 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-739cf93c-87a4-454a-9ab2-86290a976334 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:55.851272 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 936/1873] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:55.881212 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:55.922776 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739cf93c-87a4-454a-9ab2-86290a976334 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:55.924440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739cf93c-87a4-454a-9ab2-86290a976334 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:55.935971 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 938/1874] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:28:56.419617 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f203b-a28d-45eb-970d-e02764ae258a service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:56.422373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-007f203b-a28d-45eb-970d-e02764ae258a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:56.444006 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f203b-a28d-45eb-970d-e02764ae258a service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:56.450171 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e4b16-0c1e-430a-a4e4-9a8afadc58e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:56.451276 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e4b16-0c1e-430a-a4e4-9a8afadc58e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:56.451652 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e4b16-0c1e-430a-a4e4-9a8afadc58e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:56.451994 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52e4b16-0c1e-430a-a4e4-9a8afadc58e9 admin admin] PATH_INFO: `/v3/projects/056baa152e8e4753afd98ca02525e9cc` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:56.453804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e4b16-0c1e-430a-a4e4-9a8afadc58e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=056baa152e8e4753afd98ca02525e9cc)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:56.456525 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52e4b16-0c1e-430a-a4e4-9a8afadc58e9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:56.470781 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:56.529234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d52e4b16-0c1e-430a-a4e4-9a8afadc58e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '056baa152e8e4753afd98ca02525e9cc', 'request_id': 'req-d52e4b16-0c1e-430a-a4e4-9a8afadc58e9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:56.540124 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d52e4b16-0c1e-430a-a4e4-9a8afadc58e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '056baa152e8e4753afd98ca02525e9cc', 'request_id': 'req-d52e4b16-0c1e-430a-a4e4-9a8afadc58e9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:56.547517 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d52e4b16-0c1e-430a-a4e4-9a8afadc58e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '056baa152e8e4753afd98ca02525e9cc', 'request_id': 'req-d52e4b16-0c1e-430a-a4e4-9a8afadc58e9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:56.564859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d52e4b16-0c1e-430a-a4e4-9a8afadc58e9 admin admin] The token cache is being invalidate because project 056baa152e8e4753afd98ca02525e9cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:28:56.565359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d52e4b16-0c1e-430a-a4e4-9a8afadc58e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:56.566348 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d52e4b16-0c1e-430a-a4e4-9a8afadc58e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:56.569613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 937/1875] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:28:56 2022] DELETE /identity/v3/projects/056baa152e8e4753afd98ca02525e9cc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:56.572320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739cf93c-87a4-454a-9ab2-86290a976334 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:56.587686 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739cf93c-87a4-454a-9ab2-86290a976334 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:56.671410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051751d-214a-4e96-8df6-59ca45e5cec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:56.671410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051751d-214a-4e96-8df6-59ca45e5cec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:56.671410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051751d-214a-4e96-8df6-59ca45e5cec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:56.671410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051751d-214a-4e96-8df6-59ca45e5cec4 admin admin] PATH_INFO: `/v3/users/80b7089f983f4aac97b2e98cc7065008` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:56.672653 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1051751d-214a-4e96-8df6-59ca45e5cec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80b7089f983f4aac97b2e98cc7065008)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:56.685484 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1051751d-214a-4e96-8df6-59ca45e5cec4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:56.794267 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1051751d-214a-4e96-8df6-59ca45e5cec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80b7089f983f4aac97b2e98cc7065008'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:56.817063 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1051751d-214a-4e96-8df6-59ca45e5cec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80b7089f983f4aac97b2e98cc7065008'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:56.845598 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1051751d-214a-4e96-8df6-59ca45e5cec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80b7089f983f4aac97b2e98cc7065008'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:56.847675 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1051751d-214a-4e96-8df6-59ca45e5cec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80b7089f983f4aac97b2e98cc7065008'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:56.878078 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1051751d-214a-4e96-8df6-59ca45e5cec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80b7089f983f4aac97b2e98cc7065008'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:56.902378 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 939/1876] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:28:56 2022] DELETE /identity/v3/users/80b7089f983f4aac97b2e98cc7065008 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:57.425929 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e4b16-0c1e-430a-a4e4-9a8afadc58e9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:57.427402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d52e4b16-0c1e-430a-a4e4-9a8afadc58e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:57.446986 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52e4b16-0c1e-430a-a4e4-9a8afadc58e9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:57.526513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e385d4-e730-405d-acba-b1f5b97d30f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:57.527152 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e385d4-e730-405d-acba-b1f5b97d30f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:57.527433 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e385d4-e730-405d-acba-b1f5b97d30f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:57.527823 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e385d4-e730-405d-acba-b1f5b97d30f6 admin admin] PATH_INFO: `/v3/projects/c6f6bd5299e24051be854e1361b93fea` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:57.529244 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e385d4-e730-405d-acba-b1f5b97d30f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6f6bd5299e24051be854e1361b93fea)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:57.531970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e385d4-e730-405d-acba-b1f5b97d30f6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:57.600460 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a8e385d4-e730-405d-acba-b1f5b97d30f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6f6bd5299e24051be854e1361b93fea', 'request_id': 'req-a8e385d4-e730-405d-acba-b1f5b97d30f6'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:57.614074 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a8e385d4-e730-405d-acba-b1f5b97d30f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6f6bd5299e24051be854e1361b93fea', 'request_id': 'req-a8e385d4-e730-405d-acba-b1f5b97d30f6'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:57.620229 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a8e385d4-e730-405d-acba-b1f5b97d30f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6f6bd5299e24051be854e1361b93fea', 'request_id': 'req-a8e385d4-e730-405d-acba-b1f5b97d30f6'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:57.637326 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a8e385d4-e730-405d-acba-b1f5b97d30f6 admin admin] The token cache is being invalidate because project c6f6bd5299e24051be854e1361b93fea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:28:57.638016 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a8e385d4-e730-405d-acba-b1f5b97d30f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:57.639532 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a8e385d4-e730-405d-acba-b1f5b97d30f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:57.646312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 938/1877] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:28:57 2022] DELETE /identity/v3/projects/c6f6bd5299e24051be854e1361b93fea => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:57.647871 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba9eb7-dfb8-428b-a839-e296ab6ca4ea None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:57.648367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba9eb7-dfb8-428b-a839-e296ab6ca4ea None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:57.648828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba9eb7-dfb8-428b-a839-e296ab6ca4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:57.678237 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b0ba9eb7-dfb8-428b-a839-e296ab6ca4ea None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:57.680090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b0ba9eb7-dfb8-428b-a839-e296ab6ca4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:57.771937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:57.807455 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 940/1878] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:28:57.817453 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9ef88-9cc6-4177-a9ec-25a5372fb07f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:57.817994 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9ef88-9cc6-4177-a9ec-25a5372fb07f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:57.818393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b9ef88-9cc6-4177-a9ec-25a5372fb07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:57.845334 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-48b9ef88-9cc6-4177-a9ec-25a5372fb07f None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:57.847415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-48b9ef88-9cc6-4177-a9ec-25a5372fb07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:57.859072 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 939/1879] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:57.864652 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba9eb7-dfb8-428b-a839-e296ab6ca4ea None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:57.877277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba9eb7-dfb8-428b-a839-e296ab6ca4ea None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:57.885798 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497608c-381a-4cdd-af5b-9979b7b5ee2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:57.886524 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497608c-381a-4cdd-af5b-9979b7b5ee2d admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:57.887185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497608c-381a-4cdd-af5b-9979b7b5ee2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:57.887530 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497608c-381a-4cdd-af5b-9979b7b5ee2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:57.888916 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497608c-381a-4cdd-af5b-9979b7b5ee2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:28:57.889201 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497608c-381a-4cdd-af5b-9979b7b5ee2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:57.891359 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497608c-381a-4cdd-af5b-9979b7b5ee2d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:57.910167 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 941/1880] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:28:57.915018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9ef88-9cc6-4177-a9ec-25a5372fb07f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:57.928444 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b9ef88-9cc6-4177-a9ec-25a5372fb07f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:57.950268 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a983094-7f2f-4b49-87f6-ad2b4392391c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:57.951021 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a983094-7f2f-4b49-87f6-ad2b4392391c admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:57.951441 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a983094-7f2f-4b49-87f6-ad2b4392391c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:57.951686 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a983094-7f2f-4b49-87f6-ad2b4392391c admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:57.952414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a983094-7f2f-4b49-87f6-ad2b4392391c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:57.957122 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a983094-7f2f-4b49-87f6-ad2b4392391c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:58.038535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 940/1881] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:57 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:58.050138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497608c-381a-4cdd-af5b-9979b7b5ee2d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:58.065344 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497608c-381a-4cdd-af5b-9979b7b5ee2d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:58.182467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888d336-c794-422b-bcd8-df048d60acff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:58.184462 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888d336-c794-422b-bcd8-df048d60acff admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:58.184755 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888d336-c794-422b-bcd8-df048d60acff admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:58.185031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888d336-c794-422b-bcd8-df048d60acff admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:58.186117 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888d336-c794-422b-bcd8-df048d60acff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:58.186669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a983094-7f2f-4b49-87f6-ad2b4392391c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:58.188193 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888d336-c794-422b-bcd8-df048d60acff admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:58.188413 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6a983094-7f2f-4b49-87f6-ad2b4392391c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:58.197311 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:58.213435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a983094-7f2f-4b49-87f6-ad2b4392391c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:58.227855 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 942/1882] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:28:58 2022] POST /identity/v3/users => generated 291 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:58.228947 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888d336-c794-422b-bcd8-df048d60acff admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:58.237893 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68fdeb6-372a-42b4-886e-29a27740efe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:58.239206 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fdeb6-372a-42b4-886e-29a27740efe6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:58.239491 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fdeb6-372a-42b4-886e-29a27740efe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:58.239766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fdeb6-372a-42b4-886e-29a27740efe6 admin admin] PATH_INFO: `/v3/users/8848a94ad1814ff38d41931ec51925a0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:58.242321 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68fdeb6-372a-42b4-886e-29a27740efe6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8848a94ad1814ff38d41931ec51925a0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:58.244305 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68fdeb6-372a-42b4-886e-29a27740efe6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:58.250822 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888d336-c794-422b-bcd8-df048d60acff admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:58.266705 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5058d-c970-4548-937e-a1347e55e6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:58.267393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5058d-c970-4548-937e-a1347e55e6ed admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:58.267680 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5058d-c970-4548-937e-a1347e55e6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:58.268779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c5058d-c970-4548-937e-a1347e55e6ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:58.269812 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5058d-c970-4548-937e-a1347e55e6ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:58.271491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c5058d-c970-4548-937e-a1347e55e6ed admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:58.289263 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 943/1883] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:58.290450 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5058d-c970-4548-937e-a1347e55e6ed admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:58.302983 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c5058d-c970-4548-937e-a1347e55e6ed admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:58.308664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897bc1e-586e-4bc1-8fda-30e033115d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:58.309293 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897bc1e-586e-4bc1-8fda-30e033115d8a admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:58.309623 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897bc1e-586e-4bc1-8fda-30e033115d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:58.310446 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d897bc1e-586e-4bc1-8fda-30e033115d8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:58.310808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d897bc1e-586e-4bc1-8fda-30e033115d8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:58.312479 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d897bc1e-586e-4bc1-8fda-30e033115d8a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:58.329353 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 944/1884] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:58.331773 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897bc1e-586e-4bc1-8fda-30e033115d8a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:58.343696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d897bc1e-586e-4bc1-8fda-30e033115d8a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:58.352218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924847f5-87f0-47d5-8563-b61e07ead435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:58.353057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924847f5-87f0-47d5-8563-b61e07ead435 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:58.353813 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924847f5-87f0-47d5-8563-b61e07ead435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:58.354176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924847f5-87f0-47d5-8563-b61e07ead435 admin admin] PATH_INFO: `/v3/projects/60150672560343a699493671b71cb901/users/c36a79dfec484c9eb3a919ec7fddcfc7/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:58.373954 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b68fdeb6-372a-42b4-886e-29a27740efe6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8848a94ad1814ff38d41931ec51925a0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:58.375849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924847f5-87f0-47d5-8563-b61e07ead435 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60150672560343a699493671b71cb901, user_id=c36a79dfec484c9eb3a919ec7fddcfc7, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:58.379978 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924847f5-87f0-47d5-8563-b61e07ead435 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:58.388607 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b68fdeb6-372a-42b4-886e-29a27740efe6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8848a94ad1814ff38d41931ec51925a0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:58.389924 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b68fdeb6-372a-42b4-886e-29a27740efe6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8848a94ad1814ff38d41931ec51925a0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:58.391251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b68fdeb6-372a-42b4-886e-29a27740efe6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8848a94ad1814ff38d41931ec51925a0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:58.410717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b68fdeb6-372a-42b4-886e-29a27740efe6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8848a94ad1814ff38d41931ec51925a0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:58.412061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 945/1885] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:58 2022] PUT /identity/v3/projects/60150672560343a699493671b71cb901/users/c36a79dfec484c9eb3a919ec7fddcfc7/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:58.423233 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fade96-fa78-4aa1-96b2-677c6fca6598 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:58.425678 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fade96-fa78-4aa1-96b2-677c6fca6598 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:58.426498 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fade96-fa78-4aa1-96b2-677c6fca6598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:58.436047 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 941/1886] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:28:58 2022] DELETE /identity/v3/users/8848a94ad1814ff38d41931ec51925a0 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:28:58.443041 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a8dae-12de-43a6-a28c-b61953c41c47 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:58.444962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a8dae-12de-43a6-a28c-b61953c41c47 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:58.445271 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a8dae-12de-43a6-a28c-b61953c41c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:58.464493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-28fade96-fa78-4aa1-96b2-677c6fca6598 None None] MFA Rules not processed for user `c36a79dfec484c9eb3a919ec7fddcfc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:58.468393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-28fade96-fa78-4aa1-96b2-677c6fca6598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:58.481748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7c3a8dae-12de-43a6-a28c-b61953c41c47 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:58.483943 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7c3a8dae-12de-43a6-a28c-b61953c41c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:58.579246 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 942/1887] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:58.582375 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 946/1888] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:28:58.587143 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a8dae-12de-43a6-a28c-b61953c41c47 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:58.588509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7c3a8dae-12de-43a6-a28c-b61953c41c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:58.601437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a8dae-12de-43a6-a28c-b61953c41c47 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:58.606064 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e74ca2-e614-4811-af6d-3b7addd6b0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:58.606684 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e74ca2-e614-4811-af6d-3b7addd6b0fe admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:58.606972 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e74ca2-e614-4811-af6d-3b7addd6b0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:58.607247 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e74ca2-e614-4811-af6d-3b7addd6b0fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:58.607914 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e74ca2-e614-4811-af6d-3b7addd6b0fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:58.610240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e74ca2-e614-4811-af6d-3b7addd6b0fe admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:58.622701 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fade96-fa78-4aa1-96b2-677c6fca6598 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:58.624100 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-28fade96-fa78-4aa1-96b2-677c6fca6598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:58.635939 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:58.644381 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 943/1889] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:28:58 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:28:58.645558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e74ca2-e614-4811-af6d-3b7addd6b0fe admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:58.700967 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fade96-fa78-4aa1-96b2-677c6fca6598 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:58.717606 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:58.742432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e74ca2-e614-4811-af6d-3b7addd6b0fe admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:58.749087 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e865c-191d-4568-be5e-290571006613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:58.749974 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e865c-191d-4568-be5e-290571006613 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:58.750469 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e865c-191d-4568-be5e-290571006613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:58.750810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e865c-191d-4568-be5e-290571006613 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:58.751549 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e865c-191d-4568-be5e-290571006613 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:58.754515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e865c-191d-4568-be5e-290571006613 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:58.766357 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3faf1-0294-4bf6-874e-205facfe91b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:58.767544 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3faf1-0294-4bf6-874e-205facfe91b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:58.769312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3faf1-0294-4bf6-874e-205facfe91b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:58.771455 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3faf1-0294-4bf6-874e-205facfe91b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:58.785196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 944/1890] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:28:58 2022] POST /identity/v3/users => generated 297 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:28:58.786111 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3faf1-0294-4bf6-874e-205facfe91b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:58.787688 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3faf1-0294-4bf6-874e-205facfe91b0 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:58.788730 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e865c-191d-4568-be5e-290571006613 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:58.801426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e865c-191d-4568-be5e-290571006613 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:58.801953 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 947/1891] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:28:58.808907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800f588-cfb5-45cb-bb4a-b2dd075123be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:58.809555 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800f588-cfb5-45cb-bb4a-b2dd075123be admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:58.810314 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800f588-cfb5-45cb-bb4a-b2dd075123be admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:58.810314 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e800f588-cfb5-45cb-bb4a-b2dd075123be admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:58.811194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800f588-cfb5-45cb-bb4a-b2dd075123be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:58.812856 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e800f588-cfb5-45cb-bb4a-b2dd075123be admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:58.829796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 945/1892] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:58.834370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3faf1-0294-4bf6-874e-205facfe91b0 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:58.847420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3faf1-0294-4bf6-874e-205facfe91b0 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:58.853165 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e3a85-91dd-4665-bcdb-5421b91fbd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:58.854019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e3a85-91dd-4665-bcdb-5421b91fbd86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:58.854466 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e3a85-91dd-4665-bcdb-5421b91fbd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:58.854753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e3a85-91dd-4665-bcdb-5421b91fbd86 admin admin] PATH_INFO: `/v3/projects/d0fb4e8f829e45fe85e09d4a3ce2e5b3/users/57e7f4ae091a493994227e841026fb6c/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:58.872236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e3a85-91dd-4665-bcdb-5421b91fbd86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0fb4e8f829e45fe85e09d4a3ce2e5b3, user_id=57e7f4ae091a493994227e841026fb6c, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:58.875166 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66e3a85-91dd-4665-bcdb-5421b91fbd86 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:58.893988 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 948/1893] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:58 2022] PUT /identity/v3/projects/d0fb4e8f829e45fe85e09d4a3ce2e5b3/users/57e7f4ae091a493994227e841026fb6c/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:58.896234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800f588-cfb5-45cb-bb4a-b2dd075123be admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:58.911167 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e800f588-cfb5-45cb-bb4a-b2dd075123be admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:58.973092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf7ad9-8caf-4274-801c-e8dc4ceba390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:58.973778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf7ad9-8caf-4274-801c-e8dc4ceba390 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:58.974071 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf7ad9-8caf-4274-801c-e8dc4ceba390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:58.974390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf7ad9-8caf-4274-801c-e8dc4ceba390 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:58.975113 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf7ad9-8caf-4274-801c-e8dc4ceba390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:58.976984 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdf7ad9-8caf-4274-801c-e8dc4ceba390 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:58.990221 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 946/1894] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:28:58 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:28:58.993642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e3a85-91dd-4665-bcdb-5421b91fbd86 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:59.006374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e3a85-91dd-4665-bcdb-5421b91fbd86 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:59.012375 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efb02c-a1cb-431a-9931-604594087057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:59.013202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efb02c-a1cb-431a-9931-604594087057 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:59.013493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efb02c-a1cb-431a-9931-604594087057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:59.013853 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71efb02c-a1cb-431a-9931-604594087057 admin admin] PATH_INFO: `/v3/projects/d0fb4e8f829e45fe85e09d4a3ce2e5b3/users/57e7f4ae091a493994227e841026fb6c/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:59.016125 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efb02c-a1cb-431a-9931-604594087057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0fb4e8f829e45fe85e09d4a3ce2e5b3, user_id=57e7f4ae091a493994227e841026fb6c, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:59.020463 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71efb02c-a1cb-431a-9931-604594087057 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:59.037616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 949/1895] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:28:58 2022] PUT /identity/v3/projects/d0fb4e8f829e45fe85e09d4a3ce2e5b3/users/57e7f4ae091a493994227e841026fb6c/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:59.047914 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2fcba-65c8-4cdf-8229-817b4b655fd8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:59.050673 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2fcba-65c8-4cdf-8229-817b4b655fd8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:59.050913 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2fcba-65c8-4cdf-8229-817b4b655fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:59.085125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4af2fcba-65c8-4cdf-8229-817b4b655fd8 None None] MFA Rules not processed for user `57e7f4ae091a493994227e841026fb6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:28:59.087470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4af2fcba-65c8-4cdf-8229-817b4b655fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:59.189625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 947/1896] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:28:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:28:59.202108 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efb02c-a1cb-431a-9931-604594087057 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:59.204049 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-71efb02c-a1cb-431a-9931-604594087057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:59.292322 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71efb02c-a1cb-431a-9931-604594087057 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:59.297148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8e450-0556-4005-86fc-44d077c82133 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:59.298152 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8e450-0556-4005-86fc-44d077c82133 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:59.298305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8e450-0556-4005-86fc-44d077c82133 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:59.298687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8e450-0556-4005-86fc-44d077c82133 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:59.329916 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8e450-0556-4005-86fc-44d077c82133 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:59.329916 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8e450-0556-4005-86fc-44d077c82133 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:59.333359 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 950/1897] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:28:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:28:59.512873 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2fcba-65c8-4cdf-8229-817b4b655fd8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:59.514886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4af2fcba-65c8-4cdf-8229-817b4b655fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:28:59.714608 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:28:59.748254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2fcba-65c8-4cdf-8229-817b4b655fd8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:28:59.756205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5cd4d-d57f-401d-8e0e-406afd09fdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:28:59.756978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5cd4d-d57f-401d-8e0e-406afd09fdd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:28:59.757475 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5cd4d-d57f-401d-8e0e-406afd09fdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:28:59.758015 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5cd4d-d57f-401d-8e0e-406afd09fdd8 admin admin] PATH_INFO: `/v3/projects/6b7ce49305f64498aa9ae6093f637c7f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:28:59.760198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5cd4d-d57f-401d-8e0e-406afd09fdd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b7ce49305f64498aa9ae6093f637c7f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:28:59.762385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d5cd4d-d57f-401d-8e0e-406afd09fdd8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:28:59.858859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-61d5cd4d-d57f-401d-8e0e-406afd09fdd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b7ce49305f64498aa9ae6093f637c7f', 'request_id': 'req-61d5cd4d-d57f-401d-8e0e-406afd09fdd8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:59.872677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-61d5cd4d-d57f-401d-8e0e-406afd09fdd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b7ce49305f64498aa9ae6093f637c7f', 'request_id': 'req-61d5cd4d-d57f-401d-8e0e-406afd09fdd8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:59.890662 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-61d5cd4d-d57f-401d-8e0e-406afd09fdd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b7ce49305f64498aa9ae6093f637c7f', 'request_id': 'req-61d5cd4d-d57f-401d-8e0e-406afd09fdd8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:59.922129 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-61d5cd4d-d57f-401d-8e0e-406afd09fdd8 admin admin] The token cache is being invalidate because project 6b7ce49305f64498aa9ae6093f637c7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:28:59.922129 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-61d5cd4d-d57f-401d-8e0e-406afd09fdd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:59.922129 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-61d5cd4d-d57f-401d-8e0e-406afd09fdd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:28:59.927370 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 948/1898] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:28:59 2022] DELETE /identity/v3/projects/6b7ce49305f64498aa9ae6093f637c7f => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:28:59.928967 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8e450-0556-4005-86fc-44d077c82133 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:28:59.945670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8e450-0556-4005-86fc-44d077c82133 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:00.028106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:00.085052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ab7c56-0e1e-43d3-b293-b697b3b978dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:00.085762 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab7c56-0e1e-43d3-b293-b697b3b978dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:00.086048 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab7c56-0e1e-43d3-b293-b697b3b978dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:00.086329 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ab7c56-0e1e-43d3-b293-b697b3b978dc admin admin] PATH_INFO: `/v3/users/67dc008ae157470e8b9e23c4c865493d` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:00.089500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab7c56-0e1e-43d3-b293-b697b3b978dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=67dc008ae157470e8b9e23c4c865493d)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:00.091555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ab7c56-0e1e-43d3-b293-b697b3b978dc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:00.163165 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5cd4d-d57f-401d-8e0e-406afd09fdd8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:00.164772 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-61d5cd4d-d57f-401d-8e0e-406afd09fdd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:00.176669 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b0ab7c56-0e1e-43d3-b293-b697b3b978dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67dc008ae157470e8b9e23c4c865493d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:00.182529 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5cd4d-d57f-401d-8e0e-406afd09fdd8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:00.188913 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27d035-cf9c-40f5-b434-243f772c7ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:00.189015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b0ab7c56-0e1e-43d3-b293-b697b3b978dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67dc008ae157470e8b9e23c4c865493d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:00.189384 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27d035-cf9c-40f5-b434-243f772c7ef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:00.190437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27d035-cf9c-40f5-b434-243f772c7ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:00.190987 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e27d035-cf9c-40f5-b434-243f772c7ef0 admin admin] PATH_INFO: `/v3/users/c36a79dfec484c9eb3a919ec7fddcfc7` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:00.198222 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27d035-cf9c-40f5-b434-243f772c7ef0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c36a79dfec484c9eb3a919ec7fddcfc7)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:00.198222 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e27d035-cf9c-40f5-b434-243f772c7ef0 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:00.202041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b0ab7c56-0e1e-43d3-b293-b697b3b978dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67dc008ae157470e8b9e23c4c865493d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:00.203443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b0ab7c56-0e1e-43d3-b293-b697b3b978dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67dc008ae157470e8b9e23c4c865493d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:00.228477 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b0ab7c56-0e1e-43d3-b293-b697b3b978dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67dc008ae157470e8b9e23c4c865493d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:00.243270 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 951/1899] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:28:59 2022] DELETE /identity/v3/users/67dc008ae157470e8b9e23c4c865493d => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:00.292382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2e27d035-cf9c-40f5-b434-243f772c7ef0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c36a79dfec484c9eb3a919ec7fddcfc7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:00.303369 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2e27d035-cf9c-40f5-b434-243f772c7ef0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c36a79dfec484c9eb3a919ec7fddcfc7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:00.304418 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2e27d035-cf9c-40f5-b434-243f772c7ef0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c36a79dfec484c9eb3a919ec7fddcfc7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:00.305187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2e27d035-cf9c-40f5-b434-243f772c7ef0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c36a79dfec484c9eb3a919ec7fddcfc7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:00.325895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2e27d035-cf9c-40f5-b434-243f772c7ef0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c36a79dfec484c9eb3a919ec7fddcfc7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:00.368772 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 949/1900] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:29:00 2022] DELETE /identity/v3/users/c36a79dfec484c9eb3a919ec7fddcfc7 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:29:00.372014 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720970d-3901-4bb0-8e7c-2f0a1f6fd830 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:00.372425 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720970d-3901-4bb0-8e7c-2f0a1f6fd830 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:00.372803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720970d-3901-4bb0-8e7c-2f0a1f6fd830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:00.415036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b720970d-3901-4bb0-8e7c-2f0a1f6fd830 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:00.416355 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b720970d-3901-4bb0-8e7c-2f0a1f6fd830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:00.509759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 952/1901] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:00.524680 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27d035-cf9c-40f5-b434-243f772c7ef0 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:00.526736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2e27d035-cf9c-40f5-b434-243f772c7ef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:00.604028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e27d035-cf9c-40f5-b434-243f772c7ef0 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:00.608359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aec4f1-1f42-4c85-a2b6-49363af3535c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:00.609001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aec4f1-1f42-4c85-a2b6-49363af3535c service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:00.609285 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aec4f1-1f42-4c85-a2b6-49363af3535c service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:00.609601 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aec4f1-1f42-4c85-a2b6-49363af3535c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:00.621115 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aec4f1-1f42-4c85-a2b6-49363af3535c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:00.622809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aec4f1-1f42-4c85-a2b6-49363af3535c service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:00.628349 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:00.636235 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 950/1902] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:29:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:29:00.856645 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720970d-3901-4bb0-8e7c-2f0a1f6fd830 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:00.859120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b720970d-3901-4bb0-8e7c-2f0a1f6fd830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:00.915324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720970d-3901-4bb0-8e7c-2f0a1f6fd830 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:00.920880 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ab9bd-7aa7-465d-bc41-b72d21389522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:00.921806 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ab9bd-7aa7-465d-bc41-b72d21389522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:00.922245 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ab9bd-7aa7-465d-bc41-b72d21389522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:00.922600 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ab9bd-7aa7-465d-bc41-b72d21389522 admin admin] PATH_INFO: `/v3/projects/dbddbf13cfa74f609c7c06232e6f6524` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:00.924540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ab9bd-7aa7-465d-bc41-b72d21389522 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbddbf13cfa74f609c7c06232e6f6524)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:00.926623 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ab9bd-7aa7-465d-bc41-b72d21389522 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:01.004369 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:01.014551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-de4ab9bd-7aa7-465d-bc41-b72d21389522 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbddbf13cfa74f609c7c06232e6f6524', 'request_id': 'req-de4ab9bd-7aa7-465d-bc41-b72d21389522'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:01.031293 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-de4ab9bd-7aa7-465d-bc41-b72d21389522 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbddbf13cfa74f609c7c06232e6f6524', 'request_id': 'req-de4ab9bd-7aa7-465d-bc41-b72d21389522'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:01.042781 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-de4ab9bd-7aa7-465d-bc41-b72d21389522 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbddbf13cfa74f609c7c06232e6f6524', 'request_id': 'req-de4ab9bd-7aa7-465d-bc41-b72d21389522'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:01.062242 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-de4ab9bd-7aa7-465d-bc41-b72d21389522 admin admin] The token cache is being invalidate because project dbddbf13cfa74f609c7c06232e6f6524 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:29:01.062732 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-de4ab9bd-7aa7-465d-bc41-b72d21389522 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:01.063574 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-de4ab9bd-7aa7-465d-bc41-b72d21389522 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:01.068758 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 953/1903] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:29:00 2022] DELETE /identity/v3/projects/dbddbf13cfa74f609c7c06232e6f6524 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:01.070131 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aec4f1-1f42-4c85-a2b6-49363af3535c service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:01.083912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aec4f1-1f42-4c85-a2b6-49363af3535c service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:01.196263 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ab9bd-7aa7-465d-bc41-b72d21389522 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:01.197851 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-de4ab9bd-7aa7-465d-bc41-b72d21389522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:01.201562 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5141a7a-fa69-4ac0-96f7-dfa3fb20d39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:01.202613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5141a7a-fa69-4ac0-96f7-dfa3fb20d39e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:01.203681 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5141a7a-fa69-4ac0-96f7-dfa3fb20d39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:01.204106 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5141a7a-fa69-4ac0-96f7-dfa3fb20d39e admin admin] PATH_INFO: `/v3/users/f12ba0b8d7a24aa7bc046e30987df4e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:01.206655 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5141a7a-fa69-4ac0-96f7-dfa3fb20d39e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f12ba0b8d7a24aa7bc046e30987df4e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:01.208792 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5141a7a-fa69-4ac0-96f7-dfa3fb20d39e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:01.220223 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ab9bd-7aa7-465d-bc41-b72d21389522 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:01.233607 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31198505-919a-4def-893f-7daf29b380f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:01.233880 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31198505-919a-4def-893f-7daf29b380f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:01.234100 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31198505-919a-4def-893f-7daf29b380f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:01.234434 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31198505-919a-4def-893f-7daf29b380f3 admin admin] PATH_INFO: `/v3/users/b5cea747a00e4f09af2eec28663bb335` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:01.239146 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31198505-919a-4def-893f-7daf29b380f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5cea747a00e4f09af2eec28663bb335)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:01.241122 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31198505-919a-4def-893f-7daf29b380f3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:01.319006 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c5141a7a-fa69-4ac0-96f7-dfa3fb20d39e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f12ba0b8d7a24aa7bc046e30987df4e0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:01.332718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c5141a7a-fa69-4ac0-96f7-dfa3fb20d39e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f12ba0b8d7a24aa7bc046e30987df4e0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:01.333809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c5141a7a-fa69-4ac0-96f7-dfa3fb20d39e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f12ba0b8d7a24aa7bc046e30987df4e0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:01.339909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c5141a7a-fa69-4ac0-96f7-dfa3fb20d39e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f12ba0b8d7a24aa7bc046e30987df4e0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:01.345639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-31198505-919a-4def-893f-7daf29b380f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5cea747a00e4f09af2eec28663bb335'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:01.367653 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c5141a7a-fa69-4ac0-96f7-dfa3fb20d39e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f12ba0b8d7a24aa7bc046e30987df4e0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:01.371710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-31198505-919a-4def-893f-7daf29b380f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5cea747a00e4f09af2eec28663bb335'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:01.393478 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-31198505-919a-4def-893f-7daf29b380f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5cea747a00e4f09af2eec28663bb335'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:01.394410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-31198505-919a-4def-893f-7daf29b380f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5cea747a00e4f09af2eec28663bb335'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:01.395783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 951/1904] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:29:01 2022] DELETE /identity/v3/users/f12ba0b8d7a24aa7bc046e30987df4e0 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:01.397881 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5141a7a-fa69-4ac0-96f7-dfa3fb20d39e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:01.400927 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c5141a7a-fa69-4ac0-96f7-dfa3fb20d39e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:01.424876 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-31198505-919a-4def-893f-7daf29b380f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5cea747a00e4f09af2eec28663bb335'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:01.437158 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 954/1905] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:29:01 2022] DELETE /identity/v3/users/b5cea747a00e4f09af2eec28663bb335 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:29:01.442084 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e65a2-f244-446f-a68d-f915b68fa5fe None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:01.442519 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e65a2-f244-446f-a68d-f915b68fa5fe None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:01.442893 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e65a2-f244-446f-a68d-f915b68fa5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:01.485659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-1c2e65a2-f244-446f-a68d-f915b68fa5fe None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:01.489402 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1c2e65a2-f244-446f-a68d-f915b68fa5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:01.544087 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5141a7a-fa69-4ac0-96f7-dfa3fb20d39e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:01.655566 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:01.693884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 955/1906] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:29:01.714973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e65a2-f244-446f-a68d-f915b68fa5fe None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:01.717652 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1c2e65a2-f244-446f-a68d-f915b68fa5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:01.734755 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f31b0e-9356-42a6-bb4d-859440d8b7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:01.735422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f31b0e-9356-42a6-bb4d-859440d8b7ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:01.735702 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f31b0e-9356-42a6-bb4d-859440d8b7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:01.735977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f31b0e-9356-42a6-bb4d-859440d8b7ed admin admin] PATH_INFO: `/v3/projects/60150672560343a699493671b71cb901` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:01.738134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f31b0e-9356-42a6-bb4d-859440d8b7ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=60150672560343a699493671b71cb901)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:01.743893 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f31b0e-9356-42a6-bb4d-859440d8b7ed admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:01.833420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:01.840853 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d1f31b0e-9356-42a6-bb4d-859440d8b7ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60150672560343a699493671b71cb901', 'request_id': 'req-d1f31b0e-9356-42a6-bb4d-859440d8b7ed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:01.842652 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e65a2-f244-446f-a68d-f915b68fa5fe None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:01.852372 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d1f31b0e-9356-42a6-bb4d-859440d8b7ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60150672560343a699493671b71cb901', 'request_id': 'req-d1f31b0e-9356-42a6-bb4d-859440d8b7ed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:01.870357 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d1f31b0e-9356-42a6-bb4d-859440d8b7ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60150672560343a699493671b71cb901', 'request_id': 'req-d1f31b0e-9356-42a6-bb4d-859440d8b7ed'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:01.903143 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d1f31b0e-9356-42a6-bb4d-859440d8b7ed admin admin] The token cache is being invalidate because project 60150672560343a699493671b71cb901 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:29:01.903682 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d1f31b0e-9356-42a6-bb4d-859440d8b7ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:01.905233 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d1f31b0e-9356-42a6-bb4d-859440d8b7ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:01.907648 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 952/1907] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:29:01 2022] DELETE /identity/v3/projects/60150672560343a699493671b71cb901 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:01.915768 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f31b0e-9356-42a6-bb4d-859440d8b7ed admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:01.920499 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d1f31b0e-9356-42a6-bb4d-859440d8b7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:01.954967 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ee152-e397-428a-9fd0-be51380bd96e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:01.956807 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ee152-e397-428a-9fd0-be51380bd96e service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:01.957254 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ee152-e397-428a-9fd0-be51380bd96e service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:01.958505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ee152-e397-428a-9fd0-be51380bd96e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:01.960560 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3f8ee152-e397-428a-9fd0-be51380bd96e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:02.073069 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ee152-e397-428a-9fd0-be51380bd96e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:02.077196 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8ee152-e397-428a-9fd0-be51380bd96e service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:02.099895 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 956/1908] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:29:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:29:02.130048 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f31b0e-9356-42a6-bb4d-859440d8b7ed admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:02.147415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b56c43-deda-475f-a105-f0a347a00e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:02.148157 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b56c43-deda-475f-a105-f0a347a00e77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:02.149149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b56c43-deda-475f-a105-f0a347a00e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:02.157700 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ee152-e397-428a-9fd0-be51380bd96e service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:02.158817 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b56c43-deda-475f-a105-f0a347a00e77 admin admin] PATH_INFO: `/v3/users/57e7f4ae091a493994227e841026fb6c` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:02.160038 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3f8ee152-e397-428a-9fd0-be51380bd96e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:02.160581 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b56c43-deda-475f-a105-f0a347a00e77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57e7f4ae091a493994227e841026fb6c)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:02.162729 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b56c43-deda-475f-a105-f0a347a00e77 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:02.181592 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8ee152-e397-428a-9fd0-be51380bd96e service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:02.194349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89d4a4b-cb20-44ee-abff-0096736c4f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:02.195029 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d4a4b-cb20-44ee-abff-0096736c4f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:02.195318 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d4a4b-cb20-44ee-abff-0096736c4f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:02.195602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d4a4b-cb20-44ee-abff-0096736c4f4e admin admin] PATH_INFO: `/v3/projects/9374c631c31442c482ffae6c6ca5af7e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:02.197927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d4a4b-cb20-44ee-abff-0096736c4f4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9374c631c31442c482ffae6c6ca5af7e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:02.199858 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89d4a4b-cb20-44ee-abff-0096736c4f4e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:02.248869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-55b56c43-deda-475f-a105-f0a347a00e77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57e7f4ae091a493994227e841026fb6c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:02.266521 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-55b56c43-deda-475f-a105-f0a347a00e77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57e7f4ae091a493994227e841026fb6c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:02.267301 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-55b56c43-deda-475f-a105-f0a347a00e77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57e7f4ae091a493994227e841026fb6c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:02.268004 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-55b56c43-deda-475f-a105-f0a347a00e77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57e7f4ae091a493994227e841026fb6c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:02.269814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:02.293198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-55b56c43-deda-475f-a105-f0a347a00e77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57e7f4ae091a493994227e841026fb6c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:02.312268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d89d4a4b-cb20-44ee-abff-0096736c4f4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9374c631c31442c482ffae6c6ca5af7e', 'request_id': 'req-d89d4a4b-cb20-44ee-abff-0096736c4f4e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:02.323334 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 953/1909] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:29:01 2022] DELETE /identity/v3/users/57e7f4ae091a493994227e841026fb6c => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:02.335951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d89d4a4b-cb20-44ee-abff-0096736c4f4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9374c631c31442c482ffae6c6ca5af7e', 'request_id': 'req-d89d4a4b-cb20-44ee-abff-0096736c4f4e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:02.343726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d89d4a4b-cb20-44ee-abff-0096736c4f4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9374c631c31442c482ffae6c6ca5af7e', 'request_id': 'req-d89d4a4b-cb20-44ee-abff-0096736c4f4e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:02.417262 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d89d4a4b-cb20-44ee-abff-0096736c4f4e admin admin] The token cache is being invalidate because project 9374c631c31442c482ffae6c6ca5af7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:29:02.418358 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d89d4a4b-cb20-44ee-abff-0096736c4f4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:02.423540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d89d4a4b-cb20-44ee-abff-0096736c4f4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:02.429251 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 957/1910] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:29:02 2022] DELETE /identity/v3/projects/9374c631c31442c482ffae6c6ca5af7e => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:02.434600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef96de-ebe4-4c66-b992-ad61f1934123 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:02.434600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef96de-ebe4-4c66-b992-ad61f1934123 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:02.434843 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef96de-ebe4-4c66-b992-ad61f1934123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:02.482015 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-21ef96de-ebe4-4c66-b992-ad61f1934123 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:02.482015 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-21ef96de-ebe4-4c66-b992-ad61f1934123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:02.631677 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c06d7-dbd0-4c5b-8055-e21d3c1e0775 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:02.631973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c06d7-dbd0-4c5b-8055-e21d3c1e0775 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:02.632615 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c06d7-dbd0-4c5b-8055-e21d3c1e0775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:02.635425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 954/1911] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:02.657571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f40c06d7-dbd0-4c5b-8055-e21d3c1e0775 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:02.658750 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f40c06d7-dbd0-4c5b-8055-e21d3c1e0775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:02.673926 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef96de-ebe4-4c66-b992-ad61f1934123 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:02.684546 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef96de-ebe4-4c66-b992-ad61f1934123 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:02.689226 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 958/1912] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:02.701387 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f58a3-1bbb-44a7-84b5-8b0ba426c014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:02.701387 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f58a3-1bbb-44a7-84b5-8b0ba426c014 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:02.701387 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f58a3-1bbb-44a7-84b5-8b0ba426c014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:02.702312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3f58a3-1bbb-44a7-84b5-8b0ba426c014 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:02.702312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f58a3-1bbb-44a7-84b5-8b0ba426c014 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:29:02.702897 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f58a3-1bbb-44a7-84b5-8b0ba426c014 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:02.704359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3f58a3-1bbb-44a7-84b5-8b0ba426c014 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:02.728283 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c06d7-dbd0-4c5b-8055-e21d3c1e0775 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:02.732897 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 955/1913] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:29:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:29:02.739257 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c06d7-dbd0-4c5b-8055-e21d3c1e0775 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:02.744580 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91769a1-b1c5-4dff-80de-97be6fb4f7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:02.745308 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91769a1-b1c5-4dff-80de-97be6fb4f7a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:02.745624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91769a1-b1c5-4dff-80de-97be6fb4f7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:02.745897 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91769a1-b1c5-4dff-80de-97be6fb4f7a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:02.746571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91769a1-b1c5-4dff-80de-97be6fb4f7a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:02.748700 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91769a1-b1c5-4dff-80de-97be6fb4f7a2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:02.806523 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 959/1914] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:29:02 2022] POST /identity/v3/projects => generated 383 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:29:02.810723 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f58a3-1bbb-44a7-84b5-8b0ba426c014 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:02.829420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3f58a3-1bbb-44a7-84b5-8b0ba426c014 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:02.941544 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0111c-1893-4707-aa8b-49064d94a2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:02.942283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0111c-1893-4707-aa8b-49064d94a2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:02.943155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0111c-1893-4707-aa8b-49064d94a2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:02.943155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0111c-1893-4707-aa8b-49064d94a2d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:02.944141 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af0111c-1893-4707-aa8b-49064d94a2d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:02.945789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af0111c-1893-4707-aa8b-49064d94a2d2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:02.973982 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 956/1915] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:29:02 2022] POST /identity/v3/users => generated 292 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:29:02.976885 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91769a1-b1c5-4dff-80de-97be6fb4f7a2 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:02.991032 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91769a1-b1c5-4dff-80de-97be6fb4f7a2 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:02.996291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d77dfc-5739-4714-8d55-971afbf07320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:02.999055 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d77dfc-5739-4714-8d55-971afbf07320 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:02.999651 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d77dfc-5739-4714-8d55-971afbf07320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:03.000145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d77dfc-5739-4714-8d55-971afbf07320 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:03.001091 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d77dfc-5739-4714-8d55-971afbf07320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:03.003506 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d77dfc-5739-4714-8d55-971afbf07320 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:03.011289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:03.021555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 960/1916] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:02 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:29:03.022130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0111c-1893-4707-aa8b-49064d94a2d2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:03.036782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0111c-1893-4707-aa8b-49064d94a2d2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:03.039244 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d77dfc-5739-4714-8d55-971afbf07320 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:03.041045 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-76d77dfc-5739-4714-8d55-971afbf07320 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:03.053184 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee802051-649c-45d4-b249-a29aeaea80c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:03.053858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee802051-649c-45d4-b249-a29aeaea80c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:03.054141 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee802051-649c-45d4-b249-a29aeaea80c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:03.054422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee802051-649c-45d4-b249-a29aeaea80c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:03.055134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee802051-649c-45d4-b249-a29aeaea80c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:03.055902 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d77dfc-5739-4714-8d55-971afbf07320 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:03.056757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee802051-649c-45d4-b249-a29aeaea80c7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:03.061559 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90bf985-2c03-4350-8cc7-b58f80cbfd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:03.062223 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bf985-2c03-4350-8cc7-b58f80cbfd62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:03.062729 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bf985-2c03-4350-8cc7-b58f80cbfd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:03.063567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bf985-2c03-4350-8cc7-b58f80cbfd62 admin admin] PATH_INFO: `/v3/projects/d0fb4e8f829e45fe85e09d4a3ce2e5b3` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:03.066012 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90bf985-2c03-4350-8cc7-b58f80cbfd62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0fb4e8f829e45fe85e09d4a3ce2e5b3)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:03.068309 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90bf985-2c03-4350-8cc7-b58f80cbfd62 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:03.077691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 957/1917] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:29:03.079378 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee802051-649c-45d4-b249-a29aeaea80c7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:03.081826 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ee802051-649c-45d4-b249-a29aeaea80c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:03.130809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee802051-649c-45d4-b249-a29aeaea80c7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:03.132286 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b90bf985-2c03-4350-8cc7-b58f80cbfd62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0fb4e8f829e45fe85e09d4a3ce2e5b3', 'request_id': 'req-b90bf985-2c03-4350-8cc7-b58f80cbfd62'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:03.151900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b90bf985-2c03-4350-8cc7-b58f80cbfd62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0fb4e8f829e45fe85e09d4a3ce2e5b3', 'request_id': 'req-b90bf985-2c03-4350-8cc7-b58f80cbfd62'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:03.163649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b90bf985-2c03-4350-8cc7-b58f80cbfd62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0fb4e8f829e45fe85e09d4a3ce2e5b3', 'request_id': 'req-b90bf985-2c03-4350-8cc7-b58f80cbfd62'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:03.178596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b90bf985-2c03-4350-8cc7-b58f80cbfd62 admin admin] The token cache is being invalidate because project d0fb4e8f829e45fe85e09d4a3ce2e5b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:29:03.179242 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b90bf985-2c03-4350-8cc7-b58f80cbfd62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:03.180186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b90bf985-2c03-4350-8cc7-b58f80cbfd62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:03.183074 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 961/1918] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:29:03 2022] DELETE /identity/v3/projects/d0fb4e8f829e45fe85e09d4a3ce2e5b3 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:29:03.184893 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90bf985-2c03-4350-8cc7-b58f80cbfd62 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:03.188525 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b90bf985-2c03-4350-8cc7-b58f80cbfd62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:03.254182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:03.281719 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bbb320-5d88-4cd8-9a74-b21cf92de725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:03.282432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbb320-5d88-4cd8-9a74-b21cf92de725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:03.282724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbb320-5d88-4cd8-9a74-b21cf92de725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:03.283008 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bbb320-5d88-4cd8-9a74-b21cf92de725 admin admin] PATH_INFO: `/v3/projects/8b9c708b9ee44a3b9d7d49737a07c4ba` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:03.284933 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bbb320-5d88-4cd8-9a74-b21cf92de725 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b9c708b9ee44a3b9d7d49737a07c4ba)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:03.287065 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bbb320-5d88-4cd8-9a74-b21cf92de725 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:03.313832 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90bf985-2c03-4350-8cc7-b58f80cbfd62 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:03.325215 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c977014-e905-49af-bb4f-19a660fe99be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:03.326176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c977014-e905-49af-bb4f-19a660fe99be admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:03.326518 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c977014-e905-49af-bb4f-19a660fe99be admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:03.326808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c977014-e905-49af-bb4f-19a660fe99be admin admin] PATH_INFO: `/v3/projects/998659079ffe487eababb26d98288828/users/0e6f922bc9584d11847b3d904b724eba/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:03.351871 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e4bbb320-5d88-4cd8-9a74-b21cf92de725 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b9c708b9ee44a3b9d7d49737a07c4ba', 'request_id': 'req-e4bbb320-5d88-4cd8-9a74-b21cf92de725'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:03.355255 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c977014-e905-49af-bb4f-19a660fe99be admin admin] RBAC: Authorizing `identity:create_grant(project_id=998659079ffe487eababb26d98288828, user_id=0e6f922bc9584d11847b3d904b724eba, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:03.358921 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c977014-e905-49af-bb4f-19a660fe99be admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:03.361471 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e4bbb320-5d88-4cd8-9a74-b21cf92de725 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b9c708b9ee44a3b9d7d49737a07c4ba', 'request_id': 'req-e4bbb320-5d88-4cd8-9a74-b21cf92de725'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:03.368719 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e4bbb320-5d88-4cd8-9a74-b21cf92de725 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b9c708b9ee44a3b9d7d49737a07c4ba', 'request_id': 'req-e4bbb320-5d88-4cd8-9a74-b21cf92de725'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:03.378666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 962/1919] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:29:03 2022] PUT /identity/v3/projects/998659079ffe487eababb26d98288828/users/0e6f922bc9584d11847b3d904b724eba/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:03.380021 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811b03a-aed6-4f36-94a3-b0ab2a9bbdb9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:03.380312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811b03a-aed6-4f36-94a3-b0ab2a9bbdb9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:03.380592 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d811b03a-aed6-4f36-94a3-b0ab2a9bbdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:03.396102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e4bbb320-5d88-4cd8-9a74-b21cf92de725 admin admin] The token cache is being invalidate because project 8b9c708b9ee44a3b9d7d49737a07c4ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:29:03.396463 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e4bbb320-5d88-4cd8-9a74-b21cf92de725 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:03.397389 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e4bbb320-5d88-4cd8-9a74-b21cf92de725 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:03.401631 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 958/1920] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:29:03 2022] DELETE /identity/v3/projects/8b9c708b9ee44a3b9d7d49737a07c4ba => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:03.401631 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94152131-583a-4dd9-880e-2a65d40baf3b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:03.401631 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94152131-583a-4dd9-880e-2a65d40baf3b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:03.401631 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94152131-583a-4dd9-880e-2a65d40baf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:03.408019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d811b03a-aed6-4f36-94a3-b0ab2a9bbdb9 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:03.410288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d811b03a-aed6-4f36-94a3-b0ab2a9bbdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:03.435377 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-94152131-583a-4dd9-880e-2a65d40baf3b None None] MFA Rules not processed for user `0e6f922bc9584d11847b3d904b724eba`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:03.436642 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-94152131-583a-4dd9-880e-2a65d40baf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:03.499161 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 963/1921] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:03.500030 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811b03a-aed6-4f36-94a3-b0ab2a9bbdb9 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:03.502790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d811b03a-aed6-4f36-94a3-b0ab2a9bbdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:03.519694 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:03.529947 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d811b03a-aed6-4f36-94a3-b0ab2a9bbdb9 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:03.541002 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 959/1922] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:03.542336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4c858-fbc4-4370-b4ed-76ba620c9de9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:03.543522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4c858-fbc4-4370-b4ed-76ba620c9de9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:03.543798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b4c858-fbc4-4370-b4ed-76ba620c9de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:03.546844 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d1822-5457-48ec-8cd0-a57f7b038b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:03.547794 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d1822-5457-48ec-8cd0-a57f7b038b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:03.548076 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d1822-5457-48ec-8cd0-a57f7b038b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:03.548383 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d1822-5457-48ec-8cd0-a57f7b038b2e admin admin] PATH_INFO: `/v3/users/9b36c300e6424829843c56eaae049234` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:03.551007 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d1822-5457-48ec-8cd0-a57f7b038b2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b36c300e6424829843c56eaae049234)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:03.552945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d1822-5457-48ec-8cd0-a57f7b038b2e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:03.586041 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-13b4c858-fbc4-4370-b4ed-76ba620c9de9 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:03.587330 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-13b4c858-fbc4-4370-b4ed-76ba620c9de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:03.596723 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 960/1923] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:03.597612 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4c858-fbc4-4370-b4ed-76ba620c9de9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:03.598913 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-13b4c858-fbc4-4370-b4ed-76ba620c9de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:03.660632 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1b9d1822-5457-48ec-8cd0-a57f7b038b2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b36c300e6424829843c56eaae049234'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:03.680768 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1b9d1822-5457-48ec-8cd0-a57f7b038b2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b36c300e6424829843c56eaae049234'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:03.695225 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b4c858-fbc4-4370-b4ed-76ba620c9de9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:03.699163 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21477ea6-73ef-4ab3-a713-ddc7ef92ca45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:03.699799 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21477ea6-73ef-4ab3-a713-ddc7ef92ca45 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:03.700081 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21477ea6-73ef-4ab3-a713-ddc7ef92ca45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:03.700354 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21477ea6-73ef-4ab3-a713-ddc7ef92ca45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:03.700722 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1b9d1822-5457-48ec-8cd0-a57f7b038b2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b36c300e6424829843c56eaae049234'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:03.702371 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1b9d1822-5457-48ec-8cd0-a57f7b038b2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b36c300e6424829843c56eaae049234'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:03.709365 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:03.713039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21477ea6-73ef-4ab3-a713-ddc7ef92ca45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:03.714652 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21477ea6-73ef-4ab3-a713-ddc7ef92ca45 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:03.738588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1b9d1822-5457-48ec-8cd0-a57f7b038b2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b36c300e6424829843c56eaae049234'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:03.738915 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 961/1924] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:29:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:29:03.740133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21477ea6-73ef-4ab3-a713-ddc7ef92ca45 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:03.750921 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 964/1925] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:29:03 2022] DELETE /identity/v3/users/9b36c300e6424829843c56eaae049234 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:03.766933 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21477ea6-73ef-4ab3-a713-ddc7ef92ca45 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:03.896816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f7a15-a0d1-4f7f-8113-e2cf6353ac44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:03.898252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f7a15-a0d1-4f7f-8113-e2cf6353ac44 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:03.900255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f7a15-a0d1-4f7f-8113-e2cf6353ac44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:03.900729 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80f7a15-a0d1-4f7f-8113-e2cf6353ac44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:03.901695 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f7a15-a0d1-4f7f-8113-e2cf6353ac44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:29:03.902104 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f7a15-a0d1-4f7f-8113-e2cf6353ac44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:03.905685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80f7a15-a0d1-4f7f-8113-e2cf6353ac44 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:03.934612 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 962/1926] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:29:03.936670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d1822-5457-48ec-8cd0-a57f7b038b2e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:03.938495 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1b9d1822-5457-48ec-8cd0-a57f7b038b2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:03.959708 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d1822-5457-48ec-8cd0-a57f7b038b2e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:03.966521 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46c18c-183a-4a4f-9bc3-548f4ccfed8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:03.967873 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46c18c-183a-4a4f-9bc3-548f4ccfed8b admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:03.968159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46c18c-183a-4a4f-9bc3-548f4ccfed8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:03.968435 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf46c18c-183a-4a4f-9bc3-548f4ccfed8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:03.969176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46c18c-183a-4a4f-9bc3-548f4ccfed8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:03.975409 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf46c18c-183a-4a4f-9bc3-548f4ccfed8b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:04.051538 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 965/1927] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:29:03 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:29:04.056015 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f7a15-a0d1-4f7f-8113-e2cf6353ac44 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:04.071053 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80f7a15-a0d1-4f7f-8113-e2cf6353ac44 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:04.216300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b6a16-0083-4f43-8fd5-37951cc61027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:04.217602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b6a16-0083-4f43-8fd5-37951cc61027 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:04.219350 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b6a16-0083-4f43-8fd5-37951cc61027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:04.219729 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b6a16-0083-4f43-8fd5-37951cc61027 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:04.220513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b6a16-0083-4f43-8fd5-37951cc61027 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:04.225326 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b6a16-0083-4f43-8fd5-37951cc61027 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:04.305507 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 963/1928] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:29:04 2022] POST /identity/v3/users => generated 287 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:29:04.309740 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46c18c-183a-4a4f-9bc3-548f4ccfed8b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:04.324703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf46c18c-183a-4a4f-9bc3-548f4ccfed8b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:04.340087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1b8d0-1d0e-415a-b731-5f77b2bb3d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:04.340902 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1b8d0-1d0e-415a-b731-5f77b2bb3d9d admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:04.341267 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1b8d0-1d0e-415a-b731-5f77b2bb3d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:04.341777 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1b8d0-1d0e-415a-b731-5f77b2bb3d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:04.345035 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1b8d0-1d0e-415a-b731-5f77b2bb3d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:04.347496 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1b8d0-1d0e-415a-b731-5f77b2bb3d9d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:04.387768 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 966/1929] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:29:04.391932 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b6a16-0083-4f43-8fd5-37951cc61027 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:04.412551 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b6a16-0083-4f43-8fd5-37951cc61027 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:04.428361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f24cc2-05b4-4d34-8f7c-f4bd66111135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:04.429289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f24cc2-05b4-4d34-8f7c-f4bd66111135 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:04.429767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f24cc2-05b4-4d34-8f7c-f4bd66111135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:04.430385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f24cc2-05b4-4d34-8f7c-f4bd66111135 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:04.431252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f24cc2-05b4-4d34-8f7c-f4bd66111135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:04.433857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f24cc2-05b4-4d34-8f7c-f4bd66111135 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:04.447095 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 964/1930] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:29:04.451607 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1b8d0-1d0e-415a-b731-5f77b2bb3d9d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:04.474040 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1b8d0-1d0e-415a-b731-5f77b2bb3d9d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:04.496068 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843ff64-6484-4c1f-b256-1a8e6145daa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:04.500424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843ff64-6484-4c1f-b256-1a8e6145daa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:04.500874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f24cc2-05b4-4d34-8f7c-f4bd66111135 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:04.501345 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843ff64-6484-4c1f-b256-1a8e6145daa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:04.502230 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843ff64-6484-4c1f-b256-1a8e6145daa3 admin admin] PATH_INFO: `/v3/projects/60fc249e928b47f497157ed0d454d509/users/f5bcf10c1ee549c3bfb9024113e28d4e/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:04.507887 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-11f24cc2-05b4-4d34-8f7c-f4bd66111135 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:04.541591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f24cc2-05b4-4d34-8f7c-f4bd66111135 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:04.547189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843ff64-6484-4c1f-b256-1a8e6145daa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60fc249e928b47f497157ed0d454d509, user_id=f5bcf10c1ee549c3bfb9024113e28d4e, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:04.548258 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe95598-e449-438f-b27d-944ea9729dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:04.549075 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe95598-e449-438f-b27d-944ea9729dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:04.549413 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe95598-e449-438f-b27d-944ea9729dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:04.549736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe95598-e449-438f-b27d-944ea9729dad admin admin] PATH_INFO: `/v3/projects/e77b4433bf344605885778d659f601e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:04.551048 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe95598-e449-438f-b27d-944ea9729dad admin admin] RBAC: Authorizing `identity:delete_project(project_id=e77b4433bf344605885778d659f601e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:04.553514 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0843ff64-6484-4c1f-b256-1a8e6145daa3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:04.553663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe95598-e449-438f-b27d-944ea9729dad admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:04.575459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 967/1931] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:29:04 2022] PUT /identity/v3/projects/60fc249e928b47f497157ed0d454d509/users/f5bcf10c1ee549c3bfb9024113e28d4e/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:04.587664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad2773-a333-4bcd-b6f9-f95b3efbb95c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:04.588651 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad2773-a333-4bcd-b6f9-f95b3efbb95c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:04.589188 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad2773-a333-4bcd-b6f9-f95b3efbb95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:04.631277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-65ad2773-a333-4bcd-b6f9-f95b3efbb95c None None] MFA Rules not processed for user `f5bcf10c1ee549c3bfb9024113e28d4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:04.633138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-65ad2773-a333-4bcd-b6f9-f95b3efbb95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:04.642288 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cfe95598-e449-438f-b27d-944ea9729dad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e77b4433bf344605885778d659f601e0', 'request_id': 'req-cfe95598-e449-438f-b27d-944ea9729dad'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:04.659981 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cfe95598-e449-438f-b27d-944ea9729dad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e77b4433bf344605885778d659f601e0', 'request_id': 'req-cfe95598-e449-438f-b27d-944ea9729dad'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:04.668696 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cfe95598-e449-438f-b27d-944ea9729dad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e77b4433bf344605885778d659f601e0', 'request_id': 'req-cfe95598-e449-438f-b27d-944ea9729dad'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:04.671652 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:04.707189 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cfe95598-e449-438f-b27d-944ea9729dad admin admin] The token cache is being invalidate because project e77b4433bf344605885778d659f601e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:29:04.708888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cfe95598-e449-438f-b27d-944ea9729dad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:04.709954 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cfe95598-e449-438f-b27d-944ea9729dad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:04.713188 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 965/1932] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:29:04 2022] DELETE /identity/v3/projects/e77b4433bf344605885778d659f601e0 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:29:04.720457 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1fd4e-2a0d-443b-9f6a-348275598279 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:04.721753 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1fd4e-2a0d-443b-9f6a-348275598279 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:04.722894 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d1fd4e-2a0d-443b-9f6a-348275598279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:04.753496 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 968/1933] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:29:04.755169 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad2773-a333-4bcd-b6f9-f95b3efbb95c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:04.756803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-65ad2773-a333-4bcd-b6f9-f95b3efbb95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:04.777579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-36d1fd4e-2a0d-443b-9f6a-348275598279 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:04.778845 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-36d1fd4e-2a0d-443b-9f6a-348275598279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:04.803402 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:04.876426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 966/1934] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:04.886309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1156e-cae9-4507-8666-af38b5e624ee None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:04.886560 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1156e-cae9-4507-8666-af38b5e624ee None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:04.887362 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1156e-cae9-4507-8666-af38b5e624ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:04.898990 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad2773-a333-4bcd-b6f9-f95b3efbb95c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:04.911457 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-33b1156e-cae9-4507-8666-af38b5e624ee None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:04.913337 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f17de9-406b-4526-9608-cd9bd8758ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:04.913982 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f17de9-406b-4526-9608-cd9bd8758ea0 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:04.914459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f17de9-406b-4526-9608-cd9bd8758ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:04.914749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f17de9-406b-4526-9608-cd9bd8758ea0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:04.915239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-33b1156e-cae9-4507-8666-af38b5e624ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:04.916990 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f17de9-406b-4526-9608-cd9bd8758ea0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:04.919151 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f17de9-406b-4526-9608-cd9bd8758ea0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:04.930659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 967/1935] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:04.935178 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1156e-cae9-4507-8666-af38b5e624ee None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:04.950039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1156e-cae9-4507-8666-af38b5e624ee None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:04.956519 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a76e2-ee5a-4321-ba81-8d05c1d65a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:04.957638 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a76e2-ee5a-4321-ba81-8d05c1d65a11 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:04.958155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a76e2-ee5a-4321-ba81-8d05c1d65a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:04.958648 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a76e2-ee5a-4321-ba81-8d05c1d65a11 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:04.958967 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 969/1936] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:29:04 2022] POST /identity/v3/projects => generated 375 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:29:04.959798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a76e2-ee5a-4321-ba81-8d05c1d65a11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:29:04.960174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a76e2-ee5a-4321-ba81-8d05c1d65a11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:04.961976 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a76e2-ee5a-4321-ba81-8d05c1d65a11 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:04.962798 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f17de9-406b-4526-9608-cd9bd8758ea0 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:04.980433 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 968/1937] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:29:04.981517 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a76e2-ee5a-4321-ba81-8d05c1d65a11 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:04.996057 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a76e2-ee5a-4321-ba81-8d05c1d65a11 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:05.063085 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f17de9-406b-4526-9608-cd9bd8758ea0 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:05.074085 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14e6ac-33bf-4667-b6aa-e4cc7ea3d6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:05.074738 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14e6ac-33bf-4667-b6aa-e4cc7ea3d6aa admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:05.075024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14e6ac-33bf-4667-b6aa-e4cc7ea3d6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:05.075305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14e6ac-33bf-4667-b6aa-e4cc7ea3d6aa admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:05.076135 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14e6ac-33bf-4667-b6aa-e4cc7ea3d6aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:05.078439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14e6ac-33bf-4667-b6aa-e4cc7ea3d6aa admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:05.088534 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e6afb-7fc4-4d76-a900-92105a1d2e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:05.089559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e6afb-7fc4-4d76-a900-92105a1d2e14 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:05.089559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e6afb-7fc4-4d76-a900-92105a1d2e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:05.089840 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878e6afb-7fc4-4d76-a900-92105a1d2e14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:05.090849 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e6afb-7fc4-4d76-a900-92105a1d2e14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:05.093035 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878e6afb-7fc4-4d76-a900-92105a1d2e14 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:05.106857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 970/1938] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:29:04 2022] POST /identity/v3/users => generated 294 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:29:05.116790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14e6ac-33bf-4667-b6aa-e4cc7ea3d6aa admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:05.129940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14e6ac-33bf-4667-b6aa-e4cc7ea3d6aa admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:05.136116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c2542-9771-4dab-b078-1b8cc9e1a3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:05.136803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c2542-9771-4dab-b078-1b8cc9e1a3ee admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:05.137034 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c2542-9771-4dab-b078-1b8cc9e1a3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:05.137415 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847c2542-9771-4dab-b078-1b8cc9e1a3ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:05.138204 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c2542-9771-4dab-b078-1b8cc9e1a3ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:05.140033 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847c2542-9771-4dab-b078-1b8cc9e1a3ee admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:05.146244 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 969/1939] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:29:04 2022] POST /identity/v3/projects => generated 373 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:29:05.158612 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e6afb-7fc4-4d76-a900-92105a1d2e14 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:05.170336 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 971/1940] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:29:05.175139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c2542-9771-4dab-b078-1b8cc9e1a3ee admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:05.186316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847c2542-9771-4dab-b078-1b8cc9e1a3ee admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:05.231774 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:INSERT=2,keystone:DELETE=1,keystone:SELECT=97 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:05.237724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878e6afb-7fc4-4d76-a900-92105a1d2e14 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:05.245708 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0cf5d-0ebe-4228-a1f0-309bf1a8f641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:05.246807 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0cf5d-0ebe-4228-a1f0-309bf1a8f641 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:05.247781 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0cf5d-0ebe-4228-a1f0-309bf1a8f641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:05.248441 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d0cf5d-0ebe-4228-a1f0-309bf1a8f641 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:05.249793 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0cf5d-0ebe-4228-a1f0-309bf1a8f641 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:05.252750 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d0cf5d-0ebe-4228-a1f0-309bf1a8f641 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:05.254361 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af491a0d-bb41-4e26-b93f-11e2a9fcad69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:05.255038 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af491a0d-bb41-4e26-b93f-11e2a9fcad69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:05.255318 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af491a0d-bb41-4e26-b93f-11e2a9fcad69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:05.255954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af491a0d-bb41-4e26-b93f-11e2a9fcad69 admin admin] PATH_INFO: `/v3/projects/443dd294fc01418bacf0d78d528dfcef/users/6c9d8972889046c382f89a8762966b0c/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:05.274118 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af491a0d-bb41-4e26-b93f-11e2a9fcad69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=443dd294fc01418bacf0d78d528dfcef, user_id=6c9d8972889046c382f89a8762966b0c, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:05.276869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af491a0d-bb41-4e26-b93f-11e2a9fcad69 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:05.279467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:05.289717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 970/1941] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:29:05 2022] POST /identity/v3/users => generated 287 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:29:05.291608 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0cf5d-0ebe-4228-a1f0-309bf1a8f641 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:05.300594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 972/1942] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:29:05 2022] PUT /identity/v3/projects/443dd294fc01418bacf0d78d528dfcef/users/6c9d8972889046c382f89a8762966b0c/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:29:05.308687 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d0cf5d-0ebe-4228-a1f0-309bf1a8f641 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:05.315779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af491a0d-bb41-4e26-b93f-11e2a9fcad69 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:05.388720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2a441-af7d-472d-a320-c5b69bfb9f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:05.389348 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2a441-af7d-472d-a320-c5b69bfb9f18 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:05.390149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2a441-af7d-472d-a320-c5b69bfb9f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:05.391828 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2a441-af7d-472d-a320-c5b69bfb9f18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:05.393115 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d2a441-af7d-472d-a320-c5b69bfb9f18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:05.394671 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d2a441-af7d-472d-a320-c5b69bfb9f18 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:05.410724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 971/1943] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:29:05.411472 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2a441-af7d-472d-a320-c5b69bfb9f18 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:05.426174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af491a0d-bb41-4e26-b93f-11e2a9fcad69 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:05.428499 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2a441-af7d-472d-a320-c5b69bfb9f18 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:05.432432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f81ce-05c5-4218-95a9-47495166789c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:05.433077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f81ce-05c5-4218-95a9-47495166789c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:05.433454 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f81ce-05c5-4218-95a9-47495166789c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:05.434243 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7f81ce-05c5-4218-95a9-47495166789c admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:05.435047 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f81ce-05c5-4218-95a9-47495166789c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:05.435778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4433b6d-5936-4049-8f92-a6b52bd0a26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:05.436587 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4433b6d-5936-4049-8f92-a6b52bd0a26f admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:05.436741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7f81ce-05c5-4218-95a9-47495166789c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:05.437195 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4433b6d-5936-4049-8f92-a6b52bd0a26f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:05.437683 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4433b6d-5936-4049-8f92-a6b52bd0a26f admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:05.438827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4433b6d-5936-4049-8f92-a6b52bd0a26f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:05.443151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4433b6d-5936-4049-8f92-a6b52bd0a26f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:05.453125 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 973/1944] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:29:05.460563 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f81ce-05c5-4218-95a9-47495166789c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:05.468172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 972/1945] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:29:05.468172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4433b6d-5936-4049-8f92-a6b52bd0a26f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:05.476521 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7f81ce-05c5-4218-95a9-47495166789c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:05.485891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e0a2ae-404d-4046-9fcc-3432c8147da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:05.487735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0a2ae-404d-4046-9fcc-3432c8147da5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:05.488172 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0a2ae-404d-4046-9fcc-3432c8147da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:05.488758 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e0a2ae-404d-4046-9fcc-3432c8147da5 admin admin] PATH_INFO: `/v3/projects/443dd294fc01418bacf0d78d528dfcef/users/6c9d8972889046c382f89a8762966b0c/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:05.492220 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4433b6d-5936-4049-8f92-a6b52bd0a26f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:05.494492 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0a2ae-404d-4046-9fcc-3432c8147da5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=443dd294fc01418bacf0d78d528dfcef, user_id=6c9d8972889046c382f89a8762966b0c, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:05.498578 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e0a2ae-404d-4046-9fcc-3432c8147da5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:05.501388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb5833-1dad-4dde-a402-1937740dbde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:05.505038 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb5833-1dad-4dde-a402-1937740dbde5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:05.505965 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb5833-1dad-4dde-a402-1937740dbde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:05.507058 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb5833-1dad-4dde-a402-1937740dbde5 admin admin] PATH_INFO: `/v3/projects/d97f768db335416a8dcc49a7ed495cba/users/0e214d6771d245ea85fc4e3bf676fa74/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:05.529615 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 974/1946] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:29:05 2022] PUT /identity/v3/projects/443dd294fc01418bacf0d78d528dfcef/users/6c9d8972889046c382f89a8762966b0c/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:05.540866 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f188b-54d3-4f8b-a6d7-ef90d2d8e75a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:05.541552 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f188b-54d3-4f8b-a6d7-ef90d2d8e75a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:05.541866 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f188b-54d3-4f8b-a6d7-ef90d2d8e75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:05.546333 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb5833-1dad-4dde-a402-1937740dbde5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d97f768db335416a8dcc49a7ed495cba, user_id=0e214d6771d245ea85fc4e3bf676fa74, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:05.549669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb5833-1dad-4dde-a402-1937740dbde5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:05.581168 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 973/1947] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:29:05 2022] PUT /identity/v3/projects/d97f768db335416a8dcc49a7ed495cba/users/0e214d6771d245ea85fc4e3bf676fa74/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:05.593830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989c850-b8b2-4dbb-b20c-2d9d3cbb8f22 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:05.594453 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989c850-b8b2-4dbb-b20c-2d9d3cbb8f22 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:05.596409 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1989c850-b8b2-4dbb-b20c-2d9d3cbb8f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:05.613005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-658f188b-54d3-4f8b-a6d7-ef90d2d8e75a None None] MFA Rules not processed for user `6c9d8972889046c382f89a8762966b0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:05.614897 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-658f188b-54d3-4f8b-a6d7-ef90d2d8e75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:05.633046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1989c850-b8b2-4dbb-b20c-2d9d3cbb8f22 None None] MFA Rules not processed for user `0e214d6771d245ea85fc4e3bf676fa74`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:05.636764 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1989c850-b8b2-4dbb-b20c-2d9d3cbb8f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:05.732486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 975/1948] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:05.742070 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 974/1949] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:29:05.744289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f188b-54d3-4f8b-a6d7-ef90d2d8e75a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:05.747361 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-658f188b-54d3-4f8b-a6d7-ef90d2d8e75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:05.809238 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f188b-54d3-4f8b-a6d7-ef90d2d8e75a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:05.813048 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd6155-b2de-42d5-89b4-16e8dba3d989 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:05.813663 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd6155-b2de-42d5-89b4-16e8dba3d989 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:05.813951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd6155-b2de-42d5-89b4-16e8dba3d989 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:05.814238 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd6155-b2de-42d5-89b4-16e8dba3d989 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:05.828643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:05.884179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd6155-b2de-42d5-89b4-16e8dba3d989 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:05.885843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd6155-b2de-42d5-89b4-16e8dba3d989 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:05.895773 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 976/1950] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:29:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:29:06.033765 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989c850-b8b2-4dbb-b20c-2d9d3cbb8f22 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:06.035542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1989c850-b8b2-4dbb-b20c-2d9d3cbb8f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:06.048271 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1989c850-b8b2-4dbb-b20c-2d9d3cbb8f22 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:06.052747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9c1749-c428-4619-ad1b-f89ceafe57a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:06.053548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c1749-c428-4619-ad1b-f89ceafe57a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:06.053919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c1749-c428-4619-ad1b-f89ceafe57a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:06.054268 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9c1749-c428-4619-ad1b-f89ceafe57a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:06.063690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c1749-c428-4619-ad1b-f89ceafe57a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:06.066247 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9c1749-c428-4619-ad1b-f89ceafe57a9 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:06.076819 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 975/1951] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:29:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:29:06.101018 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd6155-b2de-42d5-89b4-16e8dba3d989 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:06.256922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd6155-b2de-42d5-89b4-16e8dba3d989 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:06.275311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c601c1-eb20-4fc6-9139-9adfec33ceb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:06.276235 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c601c1-eb20-4fc6-9139-9adfec33ceb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:06.276723 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c601c1-eb20-4fc6-9139-9adfec33ceb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:06.277254 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c601c1-eb20-4fc6-9139-9adfec33ceb7 admin admin] PATH_INFO: `/v3/users/0e214d6771d245ea85fc4e3bf676fa74` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:06.279437 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c601c1-eb20-4fc6-9139-9adfec33ceb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e214d6771d245ea85fc4e3bf676fa74)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:06.285585 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c601c1-eb20-4fc6-9139-9adfec33ceb7 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:06.341155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-99c601c1-eb20-4fc6-9139-9adfec33ceb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e214d6771d245ea85fc4e3bf676fa74'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:06.349773 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-99c601c1-eb20-4fc6-9139-9adfec33ceb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e214d6771d245ea85fc4e3bf676fa74'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:06.357010 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-99c601c1-eb20-4fc6-9139-9adfec33ceb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e214d6771d245ea85fc4e3bf676fa74'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:06.357699 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-99c601c1-eb20-4fc6-9139-9adfec33ceb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e214d6771d245ea85fc4e3bf676fa74'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:06.369751 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-99c601c1-eb20-4fc6-9139-9adfec33ceb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e214d6771d245ea85fc4e3bf676fa74'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:06.375672 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 977/1952] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:29:06 2022] DELETE /identity/v3/users/0e214d6771d245ea85fc4e3bf676fa74 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:06.380757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a8707-e686-400c-9a34-4e0044c97a92 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:06.381054 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a8707-e686-400c-9a34-4e0044c97a92 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:06.381341 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0a8707-e686-400c-9a34-4e0044c97a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:06.403377 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7f0a8707-e686-400c-9a34-4e0044c97a92 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:06.404615 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7f0a8707-e686-400c-9a34-4e0044c97a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:06.424074 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:06.467296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 976/1953] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:06.477057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c601c1-eb20-4fc6-9139-9adfec33ceb7 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:06.478670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-99c601c1-eb20-4fc6-9139-9adfec33ceb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:06.530312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:06.562229 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c601c1-eb20-4fc6-9139-9adfec33ceb7 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:06.566854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a34da6-19cc-4cd9-87f6-9bbcbc10ca0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:06.567624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a34da6-19cc-4cd9-87f6-9bbcbc10ca0a service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:06.567982 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a34da6-19cc-4cd9-87f6-9bbcbc10ca0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:06.568412 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a34da6-19cc-4cd9-87f6-9bbcbc10ca0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:06.581394 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a34da6-19cc-4cd9-87f6-9bbcbc10ca0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:06.582616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a34da6-19cc-4cd9-87f6-9bbcbc10ca0a service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:06.593383 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 978/1954] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:29:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:29:06.715815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a8707-e686-400c-9a34-4e0044c97a92 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:06.717276 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7f0a8707-e686-400c-9a34-4e0044c97a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:06.731164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0a8707-e686-400c-9a34-4e0044c97a92 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:06.736785 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44799e00-deb5-41b8-9329-1986047e4360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:06.737473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44799e00-deb5-41b8-9329-1986047e4360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:06.737789 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44799e00-deb5-41b8-9329-1986047e4360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:06.738066 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44799e00-deb5-41b8-9329-1986047e4360 admin admin] PATH_INFO: `/v3/users/0e6f922bc9584d11847b3d904b724eba` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:06.740040 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44799e00-deb5-41b8-9329-1986047e4360 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e6f922bc9584d11847b3d904b724eba)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:06.742432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44799e00-deb5-41b8-9329-1986047e4360 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:06.834811 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-44799e00-deb5-41b8-9329-1986047e4360 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e6f922bc9584d11847b3d904b724eba'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:06.857062 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-44799e00-deb5-41b8-9329-1986047e4360 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e6f922bc9584d11847b3d904b724eba'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:06.862922 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-44799e00-deb5-41b8-9329-1986047e4360 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e6f922bc9584d11847b3d904b724eba'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:06.867421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-44799e00-deb5-41b8-9329-1986047e4360 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e6f922bc9584d11847b3d904b724eba'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:06.921913 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-44799e00-deb5-41b8-9329-1986047e4360 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e6f922bc9584d11847b3d904b724eba'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:06.941974 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a34da6-19cc-4cd9-87f6-9bbcbc10ca0a service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:06.944235 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-76a34da6-19cc-4cd9-87f6-9bbcbc10ca0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:06.951880 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 977/1955] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:29:06 2022] DELETE /identity/v3/users/0e6f922bc9584d11847b3d904b724eba => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:06.958255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1daebef-e1da-417f-abe0-cda0fc91f4ca None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:06.958551 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1daebef-e1da-417f-abe0-cda0fc91f4ca None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:06.958809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1daebef-e1da-417f-abe0-cda0fc91f4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:06.966521 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a34da6-19cc-4cd9-87f6-9bbcbc10ca0a service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:07.020561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a1daebef-e1da-417f-abe0-cda0fc91f4ca None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:07.023994 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a1daebef-e1da-417f-abe0-cda0fc91f4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:07.098653 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d42c7e-3fd5-4714-9b74-41a2f0ea7f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:07.099708 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d42c7e-3fd5-4714-9b74-41a2f0ea7f1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:07.100170 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d42c7e-3fd5-4714-9b74-41a2f0ea7f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:07.100546 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d42c7e-3fd5-4714-9b74-41a2f0ea7f1a admin admin] PATH_INFO: `/v3/projects/d97f768db335416a8dcc49a7ed495cba` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:07.101871 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d42c7e-3fd5-4714-9b74-41a2f0ea7f1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d97f768db335416a8dcc49a7ed495cba)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:07.104061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d42c7e-3fd5-4714-9b74-41a2f0ea7f1a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:07.115472 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 978/1956] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:29:07.132281 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1daebef-e1da-417f-abe0-cda0fc91f4ca None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:07.135426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a1daebef-e1da-417f-abe0-cda0fc91f4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:07.200861 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d7d42c7e-3fd5-4714-9b74-41a2f0ea7f1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd97f768db335416a8dcc49a7ed495cba', 'request_id': 'req-d7d42c7e-3fd5-4714-9b74-41a2f0ea7f1a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:07.211613 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d7d42c7e-3fd5-4714-9b74-41a2f0ea7f1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd97f768db335416a8dcc49a7ed495cba', 'request_id': 'req-d7d42c7e-3fd5-4714-9b74-41a2f0ea7f1a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:07.213791 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:07.230395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d7d42c7e-3fd5-4714-9b74-41a2f0ea7f1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd97f768db335416a8dcc49a7ed495cba', 'request_id': 'req-d7d42c7e-3fd5-4714-9b74-41a2f0ea7f1a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:07.249938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d7d42c7e-3fd5-4714-9b74-41a2f0ea7f1a admin admin] The token cache is being invalidate because project d97f768db335416a8dcc49a7ed495cba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:29:07.250235 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d7d42c7e-3fd5-4714-9b74-41a2f0ea7f1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:07.251033 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d7d42c7e-3fd5-4714-9b74-41a2f0ea7f1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:07.252871 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1daebef-e1da-417f-abe0-cda0fc91f4ca None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:07.256805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 979/1957] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:29:06 2022] DELETE /identity/v3/projects/d97f768db335416a8dcc49a7ed495cba => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:07.258636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec60e8-acbd-4e2e-99d2-4ad007387f30 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:07.258925 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec60e8-acbd-4e2e-99d2-4ad007387f30 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:07.259198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec60e8-acbd-4e2e-99d2-4ad007387f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:07.307008 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a5ec60e8-acbd-4e2e-99d2-4ad007387f30 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:07.310068 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a5ec60e8-acbd-4e2e-99d2-4ad007387f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:07.394290 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3167ab0-e201-44c3-a447-489d53ebfd63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:07.395599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3167ab0-e201-44c3-a447-489d53ebfd63 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:07.396067 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3167ab0-e201-44c3-a447-489d53ebfd63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:07.396525 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3167ab0-e201-44c3-a447-489d53ebfd63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:07.399341 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a3167ab0-e201-44c3-a447-489d53ebfd63 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:07.427435 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 980/1958] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:07.437426 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18504f96-2564-4110-a175-e85b8a321d75 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:07.437797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18504f96-2564-4110-a175-e85b8a321d75 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:07.438027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18504f96-2564-4110-a175-e85b8a321d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:07.469343 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-18504f96-2564-4110-a175-e85b8a321d75 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:07.471180 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-18504f96-2564-4110-a175-e85b8a321d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:07.484606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 981/1959] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:29:07.487774 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18504f96-2564-4110-a175-e85b8a321d75 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:07.495544 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:07.503255 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18504f96-2564-4110-a175-e85b8a321d75 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:07.530843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4c895-4be2-4241-9b52-a3a1bf0d5de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:07.549513 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4c895-4be2-4241-9b52-a3a1bf0d5de2 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:07.549993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4c895-4be2-4241-9b52-a3a1bf0d5de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:07.550284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4c895-4be2-4241-9b52-a3a1bf0d5de2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:07.551058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4c895-4be2-4241-9b52-a3a1bf0d5de2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:29:07.551380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4c895-4be2-4241-9b52-a3a1bf0d5de2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:07.556056 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4c895-4be2-4241-9b52-a3a1bf0d5de2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:07.570227 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3167ab0-e201-44c3-a447-489d53ebfd63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:07.571690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3167ab0-e201-44c3-a447-489d53ebfd63 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:07.583314 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 982/1960] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:29:07.587734 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4c895-4be2-4241-9b52-a3a1bf0d5de2 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:07.594909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 979/1961] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:29:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:29:07.622682 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4c895-4be2-4241-9b52-a3a1bf0d5de2 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:07.628544 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59c73e-4e19-4a8e-ae3e-03e0f8c4a228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:07.629226 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59c73e-4e19-4a8e-ae3e-03e0f8c4a228 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:07.629501 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59c73e-4e19-4a8e-ae3e-03e0f8c4a228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:07.629855 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c59c73e-4e19-4a8e-ae3e-03e0f8c4a228 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:07.630564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59c73e-4e19-4a8e-ae3e-03e0f8c4a228 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:07.632554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c59c73e-4e19-4a8e-ae3e-03e0f8c4a228 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:07.677682 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 983/1962] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:29:07 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:29:07.678183 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3167ab0-e201-44c3-a447-489d53ebfd63 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:07.689059 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3167ab0-e201-44c3-a447-489d53ebfd63 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:07.763695 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f9a782-d9f6-4d68-997c-17fc625c0520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:07.764505 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9a782-d9f6-4d68-997c-17fc625c0520 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:07.764990 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9a782-d9f6-4d68-997c-17fc625c0520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:07.765301 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f9a782-d9f6-4d68-997c-17fc625c0520 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:07.770173 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f9a782-d9f6-4d68-997c-17fc625c0520 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:07.770949 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f9a782-d9f6-4d68-997c-17fc625c0520 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:07.798481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 980/1963] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:29:07 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:29:07.803341 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59c73e-4e19-4a8e-ae3e-03e0f8c4a228 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:07.816234 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c59c73e-4e19-4a8e-ae3e-03e0f8c4a228 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:07.821342 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1393d3b-f02f-4b8c-9227-785911f5d5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:07.822061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1393d3b-f02f-4b8c-9227-785911f5d5b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:07.822409 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1393d3b-f02f-4b8c-9227-785911f5d5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:07.822651 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1393d3b-f02f-4b8c-9227-785911f5d5b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:07.823430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1393d3b-f02f-4b8c-9227-785911f5d5b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:07.826267 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1393d3b-f02f-4b8c-9227-785911f5d5b4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:07.841854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 984/1964] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:29:07.843305 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f9a782-d9f6-4d68-997c-17fc625c0520 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:07.858013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f9a782-d9f6-4d68-997c-17fc625c0520 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:07.864678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ebb3e4-8b52-4108-ad73-c01c411db962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:07.865756 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebb3e4-8b52-4108-ad73-c01c411db962 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:07.866145 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebb3e4-8b52-4108-ad73-c01c411db962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:07.866428 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ebb3e4-8b52-4108-ad73-c01c411db962 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:07.867364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebb3e4-8b52-4108-ad73-c01c411db962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:07.869134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ebb3e4-8b52-4108-ad73-c01c411db962 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:07.887329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 981/1965] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:29:07.891637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1393d3b-f02f-4b8c-9227-785911f5d5b4 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:07.904455 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1393d3b-f02f-4b8c-9227-785911f5d5b4 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:07.910191 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a19f9-baff-46ab-85d1-9fca19624507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:07.910877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a19f9-baff-46ab-85d1-9fca19624507 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:07.911165 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a19f9-baff-46ab-85d1-9fca19624507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:07.911445 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633a19f9-baff-46ab-85d1-9fca19624507 admin admin] PATH_INFO: `/v3/projects/568b7b66ffdd412f8cc51834893a44b9/users/57699e010fcc490fae10db62395d36f2/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:07.930592 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a19f9-baff-46ab-85d1-9fca19624507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=568b7b66ffdd412f8cc51834893a44b9, user_id=57699e010fcc490fae10db62395d36f2, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:07.933546 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633a19f9-baff-46ab-85d1-9fca19624507 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:07.950125 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 985/1966] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:29:07 2022] PUT /identity/v3/projects/568b7b66ffdd412f8cc51834893a44b9/users/57699e010fcc490fae10db62395d36f2/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:07.962018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4956fef-1a6c-481c-b50e-1f19f8ae4bc4 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:07.962594 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4956fef-1a6c-481c-b50e-1f19f8ae4bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:07.963087 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4956fef-1a6c-481c-b50e-1f19f8ae4bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:08.010638 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a4956fef-1a6c-481c-b50e-1f19f8ae4bc4 None None] MFA Rules not processed for user `57699e010fcc490fae10db62395d36f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:08.011888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a4956fef-1a6c-481c-b50e-1f19f8ae4bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:08.024636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:08.035903 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a19f9-baff-46ab-85d1-9fca19624507 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:08.038397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-633a19f9-baff-46ab-85d1-9fca19624507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:08.114663 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633a19f9-baff-46ab-85d1-9fca19624507 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:08.120116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0b2c43-0122-46c9-a4a1-5e736ea46fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:08.120781 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b2c43-0122-46c9-a4a1-5e736ea46fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:08.121070 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b2c43-0122-46c9-a4a1-5e736ea46fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:08.121352 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0b2c43-0122-46c9-a4a1-5e736ea46fa2 admin admin] PATH_INFO: `/v3/projects/998659079ffe487eababb26d98288828` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:08.122892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b2c43-0122-46c9-a4a1-5e736ea46fa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=998659079ffe487eababb26d98288828)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:08.129543 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 982/1967] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:29:08.129781 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0b2c43-0122-46c9-a4a1-5e736ea46fa2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:08.146430 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4956fef-1a6c-481c-b50e-1f19f8ae4bc4 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:08.147734 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a4956fef-1a6c-481c-b50e-1f19f8ae4bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:08.191640 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cc0b2c43-0122-46c9-a4a1-5e736ea46fa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '998659079ffe487eababb26d98288828', 'request_id': 'req-cc0b2c43-0122-46c9-a4a1-5e736ea46fa2'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:08.202724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cc0b2c43-0122-46c9-a4a1-5e736ea46fa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '998659079ffe487eababb26d98288828', 'request_id': 'req-cc0b2c43-0122-46c9-a4a1-5e736ea46fa2'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:08.211286 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cc0b2c43-0122-46c9-a4a1-5e736ea46fa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '998659079ffe487eababb26d98288828', 'request_id': 'req-cc0b2c43-0122-46c9-a4a1-5e736ea46fa2'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:08.228631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cc0b2c43-0122-46c9-a4a1-5e736ea46fa2 admin admin] The token cache is being invalidate because project 998659079ffe487eababb26d98288828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:29:08.229769 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4956fef-1a6c-481c-b50e-1f19f8ae4bc4 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:08.229914 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cc0b2c43-0122-46c9-a4a1-5e736ea46fa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:08.230497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cc0b2c43-0122-46c9-a4a1-5e736ea46fa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:08.235455 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 986/1968] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:29:08 2022] DELETE /identity/v3/projects/998659079ffe487eababb26d98288828 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:08.237798 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074a196-5815-4e2b-bc3a-e63e4c9992cb None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:08.238093 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074a196-5815-4e2b-bc3a-e63e4c9992cb None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:08.238369 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a074a196-5815-4e2b-bc3a-e63e4c9992cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:08.259491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a074a196-5815-4e2b-bc3a-e63e4c9992cb None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:08.260719 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a074a196-5815-4e2b-bc3a-e63e4c9992cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:08.279949 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:08.298807 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52bd515-cf75-42d6-bf31-d5010d7e0aaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:08.299660 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bd515-cf75-42d6-bf31-d5010d7e0aaf service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:08.299944 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bd515-cf75-42d6-bf31-d5010d7e0aaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:08.300215 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52bd515-cf75-42d6-bf31-d5010d7e0aaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:08.303346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e52bd515-cf75-42d6-bf31-d5010d7e0aaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:08.344649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 987/1969] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:08.352936 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e572e5e-8080-47e8-8101-c14b28c760a4 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:08.353662 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e572e5e-8080-47e8-8101-c14b28c760a4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:08.354326 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e572e5e-8080-47e8-8101-c14b28c760a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:08.390352 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3e572e5e-8080-47e8-8101-c14b28c760a4 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:08.401551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3e572e5e-8080-47e8-8101-c14b28c760a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:08.401724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52bd515-cf75-42d6-bf31-d5010d7e0aaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:08.401724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52bd515-cf75-42d6-bf31-d5010d7e0aaf service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:08.411318 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 988/1970] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:29:08.415101 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 983/1971] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:29:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:29:08.416021 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e572e5e-8080-47e8-8101-c14b28c760a4 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:08.426959 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e572e5e-8080-47e8-8101-c14b28c760a4 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:08.431873 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfb922-0425-400f-8e75-528762e38b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:08.432935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfb922-0425-400f-8e75-528762e38b0f admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:08.433223 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfb922-0425-400f-8e75-528762e38b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:08.433539 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfb922-0425-400f-8e75-528762e38b0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:08.434250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfb922-0425-400f-8e75-528762e38b0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:29:08.434727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfb922-0425-400f-8e75-528762e38b0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:08.436448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfb922-0425-400f-8e75-528762e38b0f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:08.450616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 989/1972] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:29:08.455766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52bd515-cf75-42d6-bf31-d5010d7e0aaf service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:08.467455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52bd515-cf75-42d6-bf31-d5010d7e0aaf service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:08.472316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41d1a2-6a49-47af-a6e0-422a2d8ca4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:08.472960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41d1a2-6a49-47af-a6e0-422a2d8ca4c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:08.473246 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41d1a2-6a49-47af-a6e0-422a2d8ca4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:08.473549 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f41d1a2-6a49-47af-a6e0-422a2d8ca4c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:08.474256 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41d1a2-6a49-47af-a6e0-422a2d8ca4c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:08.476240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f41d1a2-6a49-47af-a6e0-422a2d8ca4c2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:08.499615 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:08.508371 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 984/1973] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:29:08 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:29:08.512366 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfb922-0425-400f-8e75-528762e38b0f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:08.524279 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfb922-0425-400f-8e75-528762e38b0f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:08.588636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb470f5a-83e0-443a-8a93-53c0d88aa3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:08.589262 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb470f5a-83e0-443a-8a93-53c0d88aa3d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:08.589569 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb470f5a-83e0-443a-8a93-53c0d88aa3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:08.589850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb470f5a-83e0-443a-8a93-53c0d88aa3d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:08.590513 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb470f5a-83e0-443a-8a93-53c0d88aa3d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:08.592431 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb470f5a-83e0-443a-8a93-53c0d88aa3d1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:08.613379 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 990/1974] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:29:08 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:29:08.617499 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41d1a2-6a49-47af-a6e0-422a2d8ca4c2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:08.632653 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f41d1a2-6a49-47af-a6e0-422a2d8ca4c2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:08.638319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55782e2-205d-4d83-b6fd-eeb81796ffbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:08.639263 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55782e2-205d-4d83-b6fd-eeb81796ffbd admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:08.639546 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55782e2-205d-4d83-b6fd-eeb81796ffbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:08.639827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55782e2-205d-4d83-b6fd-eeb81796ffbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:08.640613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55782e2-205d-4d83-b6fd-eeb81796ffbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:08.643250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55782e2-205d-4d83-b6fd-eeb81796ffbd admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:08.649452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb470f5a-83e0-443a-8a93-53c0d88aa3d1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:08.651230 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-eb470f5a-83e0-443a-8a93-53c0d88aa3d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:08.662368 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 985/1975] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:29:08.663494 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55782e2-205d-4d83-b6fd-eeb81796ffbd admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:08.666630 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb470f5a-83e0-443a-8a93-53c0d88aa3d1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:08.672521 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3019fd-4edf-44fd-a975-057d8a7a549e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:08.673173 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3019fd-4edf-44fd-a975-057d8a7a549e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:08.673465 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3019fd-4edf-44fd-a975-057d8a7a549e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:08.673893 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55782e2-205d-4d83-b6fd-eeb81796ffbd admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:08.674198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3019fd-4edf-44fd-a975-057d8a7a549e admin admin] PATH_INFO: `/v3/users/f5bcf10c1ee549c3bfb9024113e28d4e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:08.675205 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3019fd-4edf-44fd-a975-057d8a7a549e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5bcf10c1ee549c3bfb9024113e28d4e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:08.677143 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3019fd-4edf-44fd-a975-057d8a7a549e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:08.679359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8de97-a233-4921-baee-819879da25b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:08.679970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8de97-a233-4921-baee-819879da25b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:08.680253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8de97-a233-4921-baee-819879da25b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:08.680526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8de97-a233-4921-baee-819879da25b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:08.681253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8de97-a233-4921-baee-819879da25b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:08.683243 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8de97-a233-4921-baee-819879da25b6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:08.695651 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 986/1976] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:29:08.699893 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8de97-a233-4921-baee-819879da25b6 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:08.718176 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8de97-a233-4921-baee-819879da25b6 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:08.725697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539a738-c0e2-42ef-a1c7-e4f07e59568d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:08.726555 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539a738-c0e2-42ef-a1c7-e4f07e59568d admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:08.726905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539a738-c0e2-42ef-a1c7-e4f07e59568d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:08.727337 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539a738-c0e2-42ef-a1c7-e4f07e59568d admin admin] PATH_INFO: `/v3/projects/9f1b46f6a60c4fac89d0f22260e3f8d6/users/e904aa4dcc784227919619349c759472/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:08.739018 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-da3019fd-4edf-44fd-a975-057d8a7a549e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5bcf10c1ee549c3bfb9024113e28d4e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:08.745188 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3539a738-c0e2-42ef-a1c7-e4f07e59568d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f1b46f6a60c4fac89d0f22260e3f8d6, user_id=e904aa4dcc784227919619349c759472, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:08.748209 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3539a738-c0e2-42ef-a1c7-e4f07e59568d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:08.751098 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-da3019fd-4edf-44fd-a975-057d8a7a549e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5bcf10c1ee549c3bfb9024113e28d4e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:08.759901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-da3019fd-4edf-44fd-a975-057d8a7a549e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5bcf10c1ee549c3bfb9024113e28d4e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:08.760566 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-da3019fd-4edf-44fd-a975-057d8a7a549e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5bcf10c1ee549c3bfb9024113e28d4e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:08.773193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 987/1977] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:29:08 2022] PUT /identity/v3/projects/9f1b46f6a60c4fac89d0f22260e3f8d6/users/e904aa4dcc784227919619349c759472/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:08.774692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-da3019fd-4edf-44fd-a975-057d8a7a549e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5bcf10c1ee549c3bfb9024113e28d4e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:08.775873 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd516c9-0963-408b-89cf-cb1fdece4ebe None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:08.776820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd516c9-0963-408b-89cf-cb1fdece4ebe None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:08.777715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd516c9-0963-408b-89cf-cb1fdece4ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:08.781496 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 991/1978] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:29:08 2022] DELETE /identity/v3/users/f5bcf10c1ee549c3bfb9024113e28d4e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:29:08.786371 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3cd1b-b5c2-40d4-965e-132834439d73 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:08.786666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3cd1b-b5c2-40d4-965e-132834439d73 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:08.786953 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3cd1b-b5c2-40d4-965e-132834439d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:08.796100 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:08.812721 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-19e3cd1b-b5c2-40d4-965e-132834439d73 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:08.814770 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-19e3cd1b-b5c2-40d4-965e-132834439d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:08.816083 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-cbd516c9-0963-408b-89cf-cb1fdece4ebe None None] MFA Rules not processed for user `e904aa4dcc784227919619349c759472`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:08.817784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cbd516c9-0963-408b-89cf-cb1fdece4ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:08.899922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 992/1979] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:08.909203 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3cd1b-b5c2-40d4-965e-132834439d73 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:08.910672 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-19e3cd1b-b5c2-40d4-965e-132834439d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:08.916479 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 988/1980] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:29:08.928975 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd516c9-0963-408b-89cf-cb1fdece4ebe None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:08.931094 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cbd516c9-0963-408b-89cf-cb1fdece4ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:08.978241 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3cd1b-b5c2-40d4-965e-132834439d73 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:08.982020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1d43d-ec73-408c-9e6b-4e05d11148a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:08.982647 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1d43d-ec73-408c-9e6b-4e05d11148a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:08.982934 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1d43d-ec73-408c-9e6b-4e05d11148a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:08.983211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1d43d-ec73-408c-9e6b-4e05d11148a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:08.992469 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1d43d-ec73-408c-9e6b-4e05d11148a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:08.993946 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1d43d-ec73-408c-9e6b-4e05d11148a9 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:08.994208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd516c9-0963-408b-89cf-cb1fdece4ebe None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:08.997882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740013b6-6078-4dbe-b7e1-a6c503b7f12a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:08.998494 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740013b6-6078-4dbe-b7e1-a6c503b7f12a service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:08.998783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740013b6-6078-4dbe-b7e1-a6c503b7f12a service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:08.999136 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740013b6-6078-4dbe-b7e1-a6c503b7f12a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:09.003796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 993/1981] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:29:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:29:09.007770 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740013b6-6078-4dbe-b7e1-a6c503b7f12a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:09.009543 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740013b6-6078-4dbe-b7e1-a6c503b7f12a service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:09.019308 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 989/1982] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:29:08 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:29:09.325738 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1d43d-ec73-408c-9e6b-4e05d11148a9 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:09.327417 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-84a1d43d-ec73-408c-9e6b-4e05d11148a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:09.348132 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1d43d-ec73-408c-9e6b-4e05d11148a9 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:09.355123 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6c14f-948e-4b81-81dc-3ef6556b6688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:09.356057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6c14f-948e-4b81-81dc-3ef6556b6688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:09.356400 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6c14f-948e-4b81-81dc-3ef6556b6688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:09.356776 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6c14f-948e-4b81-81dc-3ef6556b6688 admin admin] PATH_INFO: `/v3/projects/60fc249e928b47f497157ed0d454d509` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:09.358267 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6c14f-948e-4b81-81dc-3ef6556b6688 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60fc249e928b47f497157ed0d454d509)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:09.360747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6c14f-948e-4b81-81dc-3ef6556b6688 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:09.426782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-caa6c14f-948e-4b81-81dc-3ef6556b6688 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60fc249e928b47f497157ed0d454d509', 'request_id': 'req-caa6c14f-948e-4b81-81dc-3ef6556b6688'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:09.435145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-caa6c14f-948e-4b81-81dc-3ef6556b6688 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60fc249e928b47f497157ed0d454d509', 'request_id': 'req-caa6c14f-948e-4b81-81dc-3ef6556b6688'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:09.444073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-caa6c14f-948e-4b81-81dc-3ef6556b6688 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60fc249e928b47f497157ed0d454d509', 'request_id': 'req-caa6c14f-948e-4b81-81dc-3ef6556b6688'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:09.457311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-caa6c14f-948e-4b81-81dc-3ef6556b6688 admin admin] The token cache is being invalidate because project 60fc249e928b47f497157ed0d454d509 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:29:09.457641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-caa6c14f-948e-4b81-81dc-3ef6556b6688 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:09.458456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-caa6c14f-948e-4b81-81dc-3ef6556b6688 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:09.461594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 994/1983] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:29:09 2022] DELETE /identity/v3/projects/60fc249e928b47f497157ed0d454d509 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:09.464341 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740013b6-6078-4dbe-b7e1-a6c503b7f12a service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:09.476017 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740013b6-6078-4dbe-b7e1-a6c503b7f12a service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:09.534676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:09.550880 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2365f76b-fd87-44d8-a1a4-b0dd8066bea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:09.551618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365f76b-fd87-44d8-a1a4-b0dd8066bea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:09.551904 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365f76b-fd87-44d8-a1a4-b0dd8066bea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:09.552180 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365f76b-fd87-44d8-a1a4-b0dd8066bea7 admin admin] PATH_INFO: `/v3/users/6c9d8972889046c382f89a8762966b0c` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:09.553383 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365f76b-fd87-44d8-a1a4-b0dd8066bea7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c9d8972889046c382f89a8762966b0c)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:09.555337 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2365f76b-fd87-44d8-a1a4-b0dd8066bea7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:09.603255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2365f76b-fd87-44d8-a1a4-b0dd8066bea7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c9d8972889046c382f89a8762966b0c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:09.610021 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2365f76b-fd87-44d8-a1a4-b0dd8066bea7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c9d8972889046c382f89a8762966b0c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:09.610686 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2365f76b-fd87-44d8-a1a4-b0dd8066bea7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c9d8972889046c382f89a8762966b0c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:09.611961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2365f76b-fd87-44d8-a1a4-b0dd8066bea7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c9d8972889046c382f89a8762966b0c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:09.623689 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2365f76b-fd87-44d8-a1a4-b0dd8066bea7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c9d8972889046c382f89a8762966b0c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:09.637135 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 990/1984] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:29:09 2022] DELETE /identity/v3/users/6c9d8972889046c382f89a8762966b0c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:09.909430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6c14f-948e-4b81-81dc-3ef6556b6688 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:09.910995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-caa6c14f-948e-4b81-81dc-3ef6556b6688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:09.929453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6c14f-948e-4b81-81dc-3ef6556b6688 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:09.961686 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:09.994929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb486eaf-2650-4bab-9fb5-8832f68003c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:09.995604 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb486eaf-2650-4bab-9fb5-8832f68003c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:09.995893 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb486eaf-2650-4bab-9fb5-8832f68003c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:09.996144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb486eaf-2650-4bab-9fb5-8832f68003c0 admin admin] PATH_INFO: `/v3/projects/443dd294fc01418bacf0d78d528dfcef` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:09.997317 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb486eaf-2650-4bab-9fb5-8832f68003c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=443dd294fc01418bacf0d78d528dfcef)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:09.999282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb486eaf-2650-4bab-9fb5-8832f68003c0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:10.050237 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cb486eaf-2650-4bab-9fb5-8832f68003c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '443dd294fc01418bacf0d78d528dfcef', 'request_id': 'req-cb486eaf-2650-4bab-9fb5-8832f68003c0'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:10.058917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cb486eaf-2650-4bab-9fb5-8832f68003c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '443dd294fc01418bacf0d78d528dfcef', 'request_id': 'req-cb486eaf-2650-4bab-9fb5-8832f68003c0'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:10.065041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cb486eaf-2650-4bab-9fb5-8832f68003c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '443dd294fc01418bacf0d78d528dfcef', 'request_id': 'req-cb486eaf-2650-4bab-9fb5-8832f68003c0'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:10.076624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cb486eaf-2650-4bab-9fb5-8832f68003c0 admin admin] The token cache is being invalidate because project 443dd294fc01418bacf0d78d528dfcef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:29:10.076913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cb486eaf-2650-4bab-9fb5-8832f68003c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:10.077582 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cb486eaf-2650-4bab-9fb5-8832f68003c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:10.079110 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 995/1985] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:29:09 2022] DELETE /identity/v3/projects/443dd294fc01418bacf0d78d528dfcef => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:10.084044 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db77fb0-a2a6-414d-841c-6f6338ab51ac None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:10.084508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db77fb0-a2a6-414d-841c-6f6338ab51ac None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:10.084717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db77fb0-a2a6-414d-841c-6f6338ab51ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:10.105418 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4db77fb0-a2a6-414d-841c-6f6338ab51ac None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:10.106714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4db77fb0-a2a6-414d-841c-6f6338ab51ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:10.172037 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 991/1986] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:10.181072 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cf27c-68ee-4b54-bb8a-412e6fa44764 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:10.181369 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cf27c-68ee-4b54-bb8a-412e6fa44764 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:10.181687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4cf27c-68ee-4b54-bb8a-412e6fa44764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:10.202451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-2c4cf27c-68ee-4b54-bb8a-412e6fa44764 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:10.203706 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2c4cf27c-68ee-4b54-bb8a-412e6fa44764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:10.213554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 996/1987] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:10.216806 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db77fb0-a2a6-414d-841c-6f6338ab51ac None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:10.226765 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db77fb0-a2a6-414d-841c-6f6338ab51ac None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:10.231386 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5afe45-fbce-498d-9340-1f0f7ee58500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:10.232028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5afe45-fbce-498d-9340-1f0f7ee58500 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:10.232300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5afe45-fbce-498d-9340-1f0f7ee58500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:10.232577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5afe45-fbce-498d-9340-1f0f7ee58500 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:10.233304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5afe45-fbce-498d-9340-1f0f7ee58500 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:29:10.233685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5afe45-fbce-498d-9340-1f0f7ee58500 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:10.235326 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5afe45-fbce-498d-9340-1f0f7ee58500 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:10.247818 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 992/1988] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:29:10.252619 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4cf27c-68ee-4b54-bb8a-412e6fa44764 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:10.264108 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4cf27c-68ee-4b54-bb8a-412e6fa44764 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:10.269598 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74330fd-ee10-47f5-9818-cfed633c7a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:10.270219 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74330fd-ee10-47f5-9818-cfed633c7a26 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:10.270515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74330fd-ee10-47f5-9818-cfed633c7a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:10.270770 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74330fd-ee10-47f5-9818-cfed633c7a26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:10.271430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74330fd-ee10-47f5-9818-cfed633c7a26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:10.273334 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74330fd-ee10-47f5-9818-cfed633c7a26 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:10.307729 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 997/1989] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:29:10 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:29:10.309059 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5afe45-fbce-498d-9340-1f0f7ee58500 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:10.319059 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5afe45-fbce-498d-9340-1f0f7ee58500 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:10.377418 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:10.389755 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af2f1b-e634-45ed-8ca5-14df68132ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:10.390797 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af2f1b-e634-45ed-8ca5-14df68132ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:10.391285 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af2f1b-e634-45ed-8ca5-14df68132ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:10.392771 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99af2f1b-e634-45ed-8ca5-14df68132ad0 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:10.393343 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af2f1b-e634-45ed-8ca5-14df68132ad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:10.395879 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99af2f1b-e634-45ed-8ca5-14df68132ad0 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:10.425898 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 993/1990] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:29:10 2022] POST /identity/v3/users => generated 292 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:29:10.428799 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74330fd-ee10-47f5-9818-cfed633c7a26 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:10.441172 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74330fd-ee10-47f5-9818-cfed633c7a26 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:10.446566 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10336b-9f67-4444-b668-f550fd94bf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:10.447201 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10336b-9f67-4444-b668-f550fd94bf8f admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:10.447488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10336b-9f67-4444-b668-f550fd94bf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:10.447759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10336b-9f67-4444-b668-f550fd94bf8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:10.448465 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10336b-9f67-4444-b668-f550fd94bf8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:10.450854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10336b-9f67-4444-b668-f550fd94bf8f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:10.464251 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 998/1991] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:29:10.468502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af2f1b-e634-45ed-8ca5-14df68132ad0 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:10.484426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99af2f1b-e634-45ed-8ca5-14df68132ad0 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:10.492825 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9ae4b-2a59-4cbb-82b4-d375e2280dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:10.494809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9ae4b-2a59-4cbb-82b4-d375e2280dbb admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:10.495170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9ae4b-2a59-4cbb-82b4-d375e2280dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:10.497641 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9ae4b-2a59-4cbb-82b4-d375e2280dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:10.498375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9ae4b-2a59-4cbb-82b4-d375e2280dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:10.500278 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9ae4b-2a59-4cbb-82b4-d375e2280dbb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:10.513621 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 994/1992] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:29:10.518632 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10336b-9f67-4444-b668-f550fd94bf8f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:10.529884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10336b-9f67-4444-b668-f550fd94bf8f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:10.542541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695eb55c-35ee-47ed-a73c-6c01f2293ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:10.549831 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695eb55c-35ee-47ed-a73c-6c01f2293ff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:10.550071 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695eb55c-35ee-47ed-a73c-6c01f2293ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:10.550354 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695eb55c-35ee-47ed-a73c-6c01f2293ff4 admin admin] PATH_INFO: `/v3/projects/da5c439035bf4fe1bcf123752ab0566f/users/9d5b147068c2491c9e4c95bbf5d0cbdd/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:10.568379 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695eb55c-35ee-47ed-a73c-6c01f2293ff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da5c439035bf4fe1bcf123752ab0566f, user_id=9d5b147068c2491c9e4c95bbf5d0cbdd, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:10.571343 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695eb55c-35ee-47ed-a73c-6c01f2293ff4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:10.587084 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 999/1993] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:29:10 2022] PUT /identity/v3/projects/da5c439035bf4fe1bcf123752ab0566f/users/9d5b147068c2491c9e4c95bbf5d0cbdd/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:29:10.598057 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e28d0-8a1e-424b-9aba-f8010a9b42f0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:10.598822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e28d0-8a1e-424b-9aba-f8010a9b42f0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:10.599262 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12e28d0-8a1e-424b-9aba-f8010a9b42f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:10.624037 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f12e28d0-8a1e-424b-9aba-f8010a9b42f0 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:10.625588 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f12e28d0-8a1e-424b-9aba-f8010a9b42f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:10.729574 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 995/1994] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:10.741774 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695eb55c-35ee-47ed-a73c-6c01f2293ff4 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:10.743741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-695eb55c-35ee-47ed-a73c-6c01f2293ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:10.807879 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695eb55c-35ee-47ed-a73c-6c01f2293ff4 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:10.811756 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91005b5e-3e09-4ec4-b2a4-6e6b4df7da38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:10.812382 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91005b5e-3e09-4ec4-b2a4-6e6b4df7da38 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:10.812671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91005b5e-3e09-4ec4-b2a4-6e6b4df7da38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:10.812950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91005b5e-3e09-4ec4-b2a4-6e6b4df7da38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:10.822091 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91005b5e-3e09-4ec4-b2a4-6e6b4df7da38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:10.823515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91005b5e-3e09-4ec4-b2a4-6e6b4df7da38 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:10.828491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:10.835050 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1000/1995] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:29:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:29:11.479967 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e6004-0d10-46fc-ba03-7bb00fe7db0a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:11.480460 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e6004-0d10-46fc-ba03-7bb00fe7db0a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:11.480550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5e6004-0d10-46fc-ba03-7bb00fe7db0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:11.510562 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-df5e6004-0d10-46fc-ba03-7bb00fe7db0a None None] MFA Rules not processed for user `9d5b147068c2491c9e4c95bbf5d0cbdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:11.511783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-df5e6004-0d10-46fc-ba03-7bb00fe7db0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:11.583836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 996/1996] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:11.594165 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91005b5e-3e09-4ec4-b2a4-6e6b4df7da38 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:11.596185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-91005b5e-3e09-4ec4-b2a4-6e6b4df7da38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:11.655789 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91005b5e-3e09-4ec4-b2a4-6e6b4df7da38 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:11.659951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575afc9-feca-4f10-a6bd-14a1db0b81a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:11.660568 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575afc9-feca-4f10-a6bd-14a1db0b81a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:11.660850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575afc9-feca-4f10-a6bd-14a1db0b81a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:11.661761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575afc9-feca-4f10-a6bd-14a1db0b81a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:11.670428 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575afc9-feca-4f10-a6bd-14a1db0b81a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:11.671862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575afc9-feca-4f10-a6bd-14a1db0b81a1 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:11.681617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1001/1997] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:29:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:29:11.721803 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e6004-0d10-46fc-ba03-7bb00fe7db0a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:11.723256 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-df5e6004-0d10-46fc-ba03-7bb00fe7db0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:11.737463 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5e6004-0d10-46fc-ba03-7bb00fe7db0a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:11.742031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ccda5-4b81-44df-bb9e-489e36943f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:11.742669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ccda5-4b81-44df-bb9e-489e36943f53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:11.742950 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ccda5-4b81-44df-bb9e-489e36943f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:11.743226 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ccda5-4b81-44df-bb9e-489e36943f53 admin admin] PATH_INFO: `/v3/users/57699e010fcc490fae10db62395d36f2` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:11.744424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43ccda5-4b81-44df-bb9e-489e36943f53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57699e010fcc490fae10db62395d36f2)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:11.746379 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43ccda5-4b81-44df-bb9e-489e36943f53 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:11.795517 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d43ccda5-4b81-44df-bb9e-489e36943f53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57699e010fcc490fae10db62395d36f2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:11.806297 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d43ccda5-4b81-44df-bb9e-489e36943f53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57699e010fcc490fae10db62395d36f2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:11.807096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d43ccda5-4b81-44df-bb9e-489e36943f53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57699e010fcc490fae10db62395d36f2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:11.807875 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d43ccda5-4b81-44df-bb9e-489e36943f53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57699e010fcc490fae10db62395d36f2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:11.809907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:11.826890 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d43ccda5-4b81-44df-bb9e-489e36943f53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57699e010fcc490fae10db62395d36f2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:11.843722 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 997/1998] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:29:11 2022] DELETE /identity/v3/users/57699e010fcc490fae10db62395d36f2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:11.848345 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91ef6a-84d7-4a9a-999b-652426c10947 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:11.848721 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91ef6a-84d7-4a9a-999b-652426c10947 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:11.848949 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91ef6a-84d7-4a9a-999b-652426c10947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:11.872767 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5a91ef6a-84d7-4a9a-999b-652426c10947 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:11.874217 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5a91ef6a-84d7-4a9a-999b-652426c10947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:11.945591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1002/1999] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:11.957947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ccda5-4b81-44df-bb9e-489e36943f53 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:11.959471 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d43ccda5-4b81-44df-bb9e-489e36943f53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:12.045608 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ccda5-4b81-44df-bb9e-489e36943f53 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:12.050199 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904c643-d030-4e6c-8eb1-de818b148faa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:12.050199 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904c643-d030-4e6c-8eb1-de818b148faa service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:12.050199 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904c643-d030-4e6c-8eb1-de818b148faa service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:12.050511 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904c643-d030-4e6c-8eb1-de818b148faa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:12.059852 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6904c643-d030-4e6c-8eb1-de818b148faa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:12.061283 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6904c643-d030-4e6c-8eb1-de818b148faa service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:12.071697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 998/2000] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:29:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:29:12.364181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91ef6a-84d7-4a9a-999b-652426c10947 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:12.365682 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5a91ef6a-84d7-4a9a-999b-652426c10947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:12.386010 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91ef6a-84d7-4a9a-999b-652426c10947 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:12.391161 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43a422a-d27e-4b00-8b48-5b838134d444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:12.391824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a422a-d27e-4b00-8b48-5b838134d444 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:12.392105 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a422a-d27e-4b00-8b48-5b838134d444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:12.393077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43a422a-d27e-4b00-8b48-5b838134d444 admin admin] PATH_INFO: `/v3/projects/568b7b66ffdd412f8cc51834893a44b9` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:12.394322 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a422a-d27e-4b00-8b48-5b838134d444 admin admin] RBAC: Authorizing `identity:delete_project(project_id=568b7b66ffdd412f8cc51834893a44b9)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:12.396240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43a422a-d27e-4b00-8b48-5b838134d444 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:12.445429 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e43a422a-d27e-4b00-8b48-5b838134d444 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '568b7b66ffdd412f8cc51834893a44b9', 'request_id': 'req-e43a422a-d27e-4b00-8b48-5b838134d444'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:12.452148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e43a422a-d27e-4b00-8b48-5b838134d444 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '568b7b66ffdd412f8cc51834893a44b9', 'request_id': 'req-e43a422a-d27e-4b00-8b48-5b838134d444'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:12.458731 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e43a422a-d27e-4b00-8b48-5b838134d444 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '568b7b66ffdd412f8cc51834893a44b9', 'request_id': 'req-e43a422a-d27e-4b00-8b48-5b838134d444'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:12.469680 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e43a422a-d27e-4b00-8b48-5b838134d444 admin admin] The token cache is being invalidate because project 568b7b66ffdd412f8cc51834893a44b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:29:12.469974 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e43a422a-d27e-4b00-8b48-5b838134d444 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:12.470648 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e43a422a-d27e-4b00-8b48-5b838134d444 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:12.472282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1003/2001] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:29:12 2022] DELETE /identity/v3/projects/568b7b66ffdd412f8cc51834893a44b9 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:12.478125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f6c96-2c01-4e2c-a829-38b4317e848b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:12.478417 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f6c96-2c01-4e2c-a829-38b4317e848b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:12.478695 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f6c96-2c01-4e2c-a829-38b4317e848b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:12.498304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-723f6c96-2c01-4e2c-a829-38b4317e848b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:12.499493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-723f6c96-2c01-4e2c-a829-38b4317e848b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:12.560587 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 999/2002] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:29:12.569379 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df0b89-84a5-4fc8-b117-59bc54db5f36 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:12.569700 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df0b89-84a5-4fc8-b117-59bc54db5f36 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:12.569984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df0b89-84a5-4fc8-b117-59bc54db5f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:12.593759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-31df0b89-84a5-4fc8-b117-59bc54db5f36 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:12.594992 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-31df0b89-84a5-4fc8-b117-59bc54db5f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:12.603856 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1004/2003] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:29:12.608220 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f6c96-2c01-4e2c-a829-38b4317e848b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:12.620663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f6c96-2c01-4e2c-a829-38b4317e848b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:12.627698 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb55a2-eb20-499d-881f-90e1336be4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:12.628260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb55a2-eb20-499d-881f-90e1336be4e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:12.628981 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb55a2-eb20-499d-881f-90e1336be4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:12.629271 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bb55a2-eb20-499d-881f-90e1336be4e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:12.630150 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb55a2-eb20-499d-881f-90e1336be4e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:29:12.630524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb55a2-eb20-499d-881f-90e1336be4e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:12.632189 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bb55a2-eb20-499d-881f-90e1336be4e7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:12.645837 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1000/2004] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:29:12.650384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df0b89-84a5-4fc8-b117-59bc54db5f36 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:12.659715 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df0b89-84a5-4fc8-b117-59bc54db5f36 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:12.665177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8bc46a-025a-4590-b9ef-d09784602975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:12.665841 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bc46a-025a-4590-b9ef-d09784602975 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:12.666128 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bc46a-025a-4590-b9ef-d09784602975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:12.666500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bc46a-025a-4590-b9ef-d09784602975 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:12.667094 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8bc46a-025a-4590-b9ef-d09784602975 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:12.669000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8bc46a-025a-4590-b9ef-d09784602975 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:12.695475 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:12.703859 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1005/2005] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:29:12 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:29:12.708088 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb55a2-eb20-499d-881f-90e1336be4e7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:12.718109 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bb55a2-eb20-499d-881f-90e1336be4e7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:12.774811 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439d7e0-acfc-4198-b739-6e538552c4a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:12.775455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439d7e0-acfc-4198-b739-6e538552c4a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:12.775735 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439d7e0-acfc-4198-b739-6e538552c4a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:12.776005 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439d7e0-acfc-4198-b739-6e538552c4a5 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:12.776693 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439d7e0-acfc-4198-b739-6e538552c4a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:12.778665 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439d7e0-acfc-4198-b739-6e538552c4a5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:12.788237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:12.804277 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1001/2006] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:29:12 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:29:12.805579 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8bc46a-025a-4590-b9ef-d09784602975 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:12.816339 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8bc46a-025a-4590-b9ef-d09784602975 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:12.821134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb93c3-c308-4f61-8b7d-1806dd9baac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:12.821779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb93c3-c308-4f61-8b7d-1806dd9baac2 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:12.822056 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb93c3-c308-4f61-8b7d-1806dd9baac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:12.822356 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bb93c3-c308-4f61-8b7d-1806dd9baac2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:12.823051 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb93c3-c308-4f61-8b7d-1806dd9baac2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:12.824644 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bb93c3-c308-4f61-8b7d-1806dd9baac2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:12.835151 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1006/2007] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:29:12.839304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439d7e0-acfc-4198-b739-6e538552c4a5 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:12.849791 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439d7e0-acfc-4198-b739-6e538552c4a5 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:12.854814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c0e8a-031d-4ce8-a015-ae5aae74921f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:12.855479 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c0e8a-031d-4ce8-a015-ae5aae74921f admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:12.855812 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c0e8a-031d-4ce8-a015-ae5aae74921f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:12.856155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c0e8a-031d-4ce8-a015-ae5aae74921f admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:12.856937 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c0e8a-031d-4ce8-a015-ae5aae74921f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:12.858934 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c0e8a-031d-4ce8-a015-ae5aae74921f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:12.870640 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1002/2008] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:29:12.874728 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb93c3-c308-4f61-8b7d-1806dd9baac2 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:12.885469 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bb93c3-c308-4f61-8b7d-1806dd9baac2 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:12.890241 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d955be-9880-412a-8eb3-652ae2d595aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:12.890923 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d955be-9880-412a-8eb3-652ae2d595aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:12.891207 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d955be-9880-412a-8eb3-652ae2d595aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:12.891481 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d955be-9880-412a-8eb3-652ae2d595aa admin admin] PATH_INFO: `/v3/projects/cece5f52f70a4d65939e5387110b8a83/users/9e3b2ad460134d52b9fd514e783a6878/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:12.905807 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d955be-9880-412a-8eb3-652ae2d595aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=cece5f52f70a4d65939e5387110b8a83, user_id=9e3b2ad460134d52b9fd514e783a6878, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:12.909693 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d955be-9880-412a-8eb3-652ae2d595aa admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:12.922355 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1007/2009] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:29:12 2022] PUT /identity/v3/projects/cece5f52f70a4d65939e5387110b8a83/users/9e3b2ad460134d52b9fd514e783a6878/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:12.934857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401b53b-5d7f-4522-9a76-4bcefa24600a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:12.935142 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401b53b-5d7f-4522-9a76-4bcefa24600a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:12.935420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401b53b-5d7f-4522-9a76-4bcefa24600a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:12.963728 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3401b53b-5d7f-4522-9a76-4bcefa24600a None None] MFA Rules not processed for user `9e3b2ad460134d52b9fd514e783a6878`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:12.964930 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3401b53b-5d7f-4522-9a76-4bcefa24600a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:13.031005 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1003/2010] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:13.050797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d955be-9880-412a-8eb3-652ae2d595aa admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:13.052642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c6d955be-9880-412a-8eb3-652ae2d595aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:13.101573 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7f4ed-97ed-484e-91dc-972541cff568 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:13.101897 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7f4ed-97ed-484e-91dc-972541cff568 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:13.102179 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f7f4ed-97ed-484e-91dc-972541cff568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:13.119506 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d955be-9880-412a-8eb3-652ae2d595aa admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:13.123322 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13c5e5-2b41-4795-ba03-97d801c9979a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:13.123951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13c5e5-2b41-4795-ba03-97d801c9979a service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:13.124236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13c5e5-2b41-4795-ba03-97d801c9979a service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:13.124518 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13c5e5-2b41-4795-ba03-97d801c9979a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:13.126839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-26f7f4ed-97ed-484e-91dc-972541cff568 None None] MFA Rules not processed for user `9d5b147068c2491c9e4c95bbf5d0cbdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:13.128111 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-26f7f4ed-97ed-484e-91dc-972541cff568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:13.134123 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c13c5e5-2b41-4795-ba03-97d801c9979a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:13.135635 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c13c5e5-2b41-4795-ba03-97d801c9979a service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:13.154070 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1008/2011] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:29:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:29:13.192757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1004/2012] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:29:23.140977 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:23.182563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:INSERT=1,keystone:SELECT=64 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:26.640628 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13c5e5-2b41-4795-ba03-97d801c9979a service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:26.714665 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13c5e5-2b41-4795-ba03-97d801c9979a service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:26.728670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a14bf-1387-453a-a118-8c72b99f9d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:26.729407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a14bf-1387-453a-a118-8c72b99f9d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:26.729806 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a14bf-1387-453a-a118-8c72b99f9d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:26.730145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a14bf-1387-453a-a118-8c72b99f9d09 admin admin] PATH_INFO: `/v3/users/9e3b2ad460134d52b9fd514e783a6878` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:26.732082 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a14bf-1387-453a-a118-8c72b99f9d09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e3b2ad460134d52b9fd514e783a6878)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:26.734795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a14bf-1387-453a-a118-8c72b99f9d09 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:26.798307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-509a14bf-1387-453a-a118-8c72b99f9d09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e3b2ad460134d52b9fd514e783a6878'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:26.809985 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-509a14bf-1387-453a-a118-8c72b99f9d09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e3b2ad460134d52b9fd514e783a6878'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:26.819798 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-509a14bf-1387-453a-a118-8c72b99f9d09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e3b2ad460134d52b9fd514e783a6878'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:26.820661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-509a14bf-1387-453a-a118-8c72b99f9d09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e3b2ad460134d52b9fd514e783a6878'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:26.837380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-509a14bf-1387-453a-a118-8c72b99f9d09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e3b2ad460134d52b9fd514e783a6878'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:26.848666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1009/2013] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:29:26 2022] DELETE /identity/v3/users/9e3b2ad460134d52b9fd514e783a6878 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:26.850504 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1478b4-9b2f-4cdf-a5ae-d6b6181f1ff5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:26.851032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1478b4-9b2f-4cdf-a5ae-d6b6181f1ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:26.851032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1478b4-9b2f-4cdf-a5ae-d6b6181f1ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:26.893009 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1d1478b4-9b2f-4cdf-a5ae-d6b6181f1ff5 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:26.894507 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1d1478b4-9b2f-4cdf-a5ae-d6b6181f1ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:26.974316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1005/2014] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:26.984223 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a14bf-1387-453a-a118-8c72b99f9d09 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:26.986248 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-509a14bf-1387-453a-a118-8c72b99f9d09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:27.046595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a14bf-1387-453a-a118-8c72b99f9d09 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:27.051538 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569ecc5-3d30-4ee9-86fb-5cdf88c255c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:27.052176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569ecc5-3d30-4ee9-86fb-5cdf88c255c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:27.052460 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569ecc5-3d30-4ee9-86fb-5cdf88c255c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:27.052737 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569ecc5-3d30-4ee9-86fb-5cdf88c255c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:27.062800 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569ecc5-3d30-4ee9-86fb-5cdf88c255c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:27.065448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569ecc5-3d30-4ee9-86fb-5cdf88c255c7 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:27.075296 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1010/2015] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:29:27.412076 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1478b4-9b2f-4cdf-a5ae-d6b6181f1ff5 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:27.415564 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1d1478b4-9b2f-4cdf-a5ae-d6b6181f1ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:27.434354 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1478b4-9b2f-4cdf-a5ae-d6b6181f1ff5 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:27.466309 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7049f-e449-42b1-987f-42d02bb4c92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:27.466996 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7049f-e449-42b1-987f-42d02bb4c92f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:27.467360 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7049f-e449-42b1-987f-42d02bb4c92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:27.467647 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7049f-e449-42b1-987f-42d02bb4c92f admin admin] PATH_INFO: `/v3/projects/cece5f52f70a4d65939e5387110b8a83` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:27.468937 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7049f-e449-42b1-987f-42d02bb4c92f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cece5f52f70a4d65939e5387110b8a83)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:27.471877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7049f-e449-42b1-987f-42d02bb4c92f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:27.543375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-82a7049f-e449-42b1-987f-42d02bb4c92f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cece5f52f70a4d65939e5387110b8a83', 'request_id': 'req-82a7049f-e449-42b1-987f-42d02bb4c92f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:27.550863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-82a7049f-e449-42b1-987f-42d02bb4c92f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cece5f52f70a4d65939e5387110b8a83', 'request_id': 'req-82a7049f-e449-42b1-987f-42d02bb4c92f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:27.557114 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-82a7049f-e449-42b1-987f-42d02bb4c92f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cece5f52f70a4d65939e5387110b8a83', 'request_id': 'req-82a7049f-e449-42b1-987f-42d02bb4c92f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:27.578426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-82a7049f-e449-42b1-987f-42d02bb4c92f admin admin] The token cache is being invalidate because project cece5f52f70a4d65939e5387110b8a83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:29:27.578708 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-82a7049f-e449-42b1-987f-42d02bb4c92f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:27.580205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-82a7049f-e449-42b1-987f-42d02bb4c92f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:27.582208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1006/2016] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:29:27 2022] DELETE /identity/v3/projects/cece5f52f70a4d65939e5387110b8a83 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:27.586975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24aa58-78cc-439f-bac3-309c25875b33 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:27.587289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24aa58-78cc-439f-bac3-309c25875b33 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:27.587583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24aa58-78cc-439f-bac3-309c25875b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:27.607745 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-bc24aa58-78cc-439f-bac3-309c25875b33 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:27.608973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bc24aa58-78cc-439f-bac3-309c25875b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:27.630697 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:27.686531 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1011/2017] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:27.695825 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b242e-ef9f-4462-8750-235084132f21 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:27.696119 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b242e-ef9f-4462-8750-235084132f21 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:27.696459 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99b242e-ef9f-4462-8750-235084132f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:27.722883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c99b242e-ef9f-4462-8750-235084132f21 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:27.724201 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c99b242e-ef9f-4462-8750-235084132f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:27.738261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1007/2018] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:27.742649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24aa58-78cc-439f-bac3-309c25875b33 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:27.753827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24aa58-78cc-439f-bac3-309c25875b33 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:27.759264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f52d6-586f-4ca7-aeb3-e96185d33cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:27.759892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f52d6-586f-4ca7-aeb3-e96185d33cb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:27.760546 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f52d6-586f-4ca7-aeb3-e96185d33cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:27.760546 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f52d6-586f-4ca7-aeb3-e96185d33cb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:27.761156 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f52d6-586f-4ca7-aeb3-e96185d33cb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:29:27.761469 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f52d6-586f-4ca7-aeb3-e96185d33cb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:27.763226 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f52d6-586f-4ca7-aeb3-e96185d33cb6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:27.777281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1012/2019] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:29:27.781487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b242e-ef9f-4462-8750-235084132f21 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:27.791429 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99b242e-ef9f-4462-8750-235084132f21 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:27.796794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1f7b5-e512-4706-bcde-582ba755c235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:27.797438 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1f7b5-e512-4706-bcde-582ba755c235 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:27.798288 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1f7b5-e512-4706-bcde-582ba755c235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:27.798653 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1f7b5-e512-4706-bcde-582ba755c235 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:27.799345 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1f7b5-e512-4706-bcde-582ba755c235 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:27.801290 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1f7b5-e512-4706-bcde-582ba755c235 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:27.836189 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1008/2020] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:29:27 2022] POST /identity/v3/projects => generated 397 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:29:27.837425 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f52d6-586f-4ca7-aeb3-e96185d33cb6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:27.848608 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f52d6-586f-4ca7-aeb3-e96185d33cb6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:27.912965 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9dead1-0262-498a-8839-a4f0b9c7818d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:27.913805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9dead1-0262-498a-8839-a4f0b9c7818d admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:27.914221 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9dead1-0262-498a-8839-a4f0b9c7818d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:27.914502 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9dead1-0262-498a-8839-a4f0b9c7818d admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:27.915176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9dead1-0262-498a-8839-a4f0b9c7818d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:27.917233 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9dead1-0262-498a-8839-a4f0b9c7818d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:27.960954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1013/2021] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:29:27 2022] POST /identity/v3/users => generated 299 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:29:27.963783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1f7b5-e512-4706-bcde-582ba755c235 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:27.979570 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1f7b5-e512-4706-bcde-582ba755c235 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:27.993867 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f1f9c-8ba5-444a-885b-aafa543eb9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:27.993867 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f1f9c-8ba5-444a-885b-aafa543eb9e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:27.993867 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f1f9c-8ba5-444a-885b-aafa543eb9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:27.993867 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f1f9c-8ba5-444a-885b-aafa543eb9e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:27.994218 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f1f9c-8ba5-444a-885b-aafa543eb9e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:27.994218 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f1f9c-8ba5-444a-885b-aafa543eb9e7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:28.003420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1009/2022] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:29:28.007644 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9dead1-0262-498a-8839-a4f0b9c7818d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:28.018334 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9dead1-0262-498a-8839-a4f0b9c7818d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:28.023910 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f20792-e3e4-4358-a83e-c1f0422d0f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:28.024087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f20792-e3e4-4358-a83e-c1f0422d0f1d admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:28.024362 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f20792-e3e4-4358-a83e-c1f0422d0f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:28.024641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f20792-e3e4-4358-a83e-c1f0422d0f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:28.025558 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f20792-e3e4-4358-a83e-c1f0422d0f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:28.029113 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f20792-e3e4-4358-a83e-c1f0422d0f1d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:28.047082 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1014/2023] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:29:28.058550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f1f9c-8ba5-444a-885b-aafa543eb9e7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:28.074839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f1f9c-8ba5-444a-885b-aafa543eb9e7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:28.080107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079ffed-6f0b-496f-b4c0-5e1463c9602a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:28.081060 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079ffed-6f0b-496f-b4c0-5e1463c9602a admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:28.081341 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079ffed-6f0b-496f-b4c0-5e1463c9602a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:28.081661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079ffed-6f0b-496f-b4c0-5e1463c9602a admin admin] PATH_INFO: `/v3/projects/4a853b6310d846a992dcab02c51a4057/users/08bd37c3d9a04406b33c04be17deeb04/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:28.113047 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079ffed-6f0b-496f-b4c0-5e1463c9602a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a853b6310d846a992dcab02c51a4057, user_id=08bd37c3d9a04406b33c04be17deeb04, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:28.116303 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079ffed-6f0b-496f-b4c0-5e1463c9602a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:28.141569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1010/2024] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:29:28 2022] PUT /identity/v3/projects/4a853b6310d846a992dcab02c51a4057/users/08bd37c3d9a04406b33c04be17deeb04/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:28.146948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53dbad6-bdfc-46ac-82b9-72023a875450 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:28.147348 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53dbad6-bdfc-46ac-82b9-72023a875450 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:28.147725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53dbad6-bdfc-46ac-82b9-72023a875450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:28.180735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c53dbad6-bdfc-46ac-82b9-72023a875450 None None] MFA Rules not processed for user `08bd37c3d9a04406b33c04be17deeb04`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:28.182183 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c53dbad6-bdfc-46ac-82b9-72023a875450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:28.281582 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1015/2025] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:29:28.289889 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079ffed-6f0b-496f-b4c0-5e1463c9602a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:28.291975 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a079ffed-6f0b-496f-b4c0-5e1463c9602a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:28.322429 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:28.378986 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079ffed-6f0b-496f-b4c0-5e1463c9602a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:28.388385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484f9f7b-4fc4-46c7-89c2-e3f7764c3113 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:28.389356 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f9f7b-4fc4-46c7-89c2-e3f7764c3113 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:28.389518 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f9f7b-4fc4-46c7-89c2-e3f7764c3113 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:28.389920 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484f9f7b-4fc4-46c7-89c2-e3f7764c3113 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:28.400736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f9f7b-4fc4-46c7-89c2-e3f7764c3113 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:28.401947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484f9f7b-4fc4-46c7-89c2-e3f7764c3113 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:28.413120 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1011/2026] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:29:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:29:29.252570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24f1aa-e3a4-4f15-ac52-672f8e661d0f None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:29.253152 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24f1aa-e3a4-4f15-ac52-672f8e661d0f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:29.253152 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a24f1aa-e3a4-4f15-ac52-672f8e661d0f None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:29.254589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1016/2027] 10.222.0.39 () {54 vars in 921 bytes} [Thu Jul 28 22:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 28 22:29:29.299480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01dc48-8168-4d16-97f6-d65a82f92523 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:29.300007 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01dc48-8168-4d16-97f6-d65a82f92523 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:29.300091 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b01dc48-8168-4d16-97f6-d65a82f92523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:29.323743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8b01dc48-8168-4d16-97f6-d65a82f92523 None None] MFA Rules not processed for user `fe003cd7bfbe4b92b1d64d300c3e7d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:29.324973 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8b01dc48-8168-4d16-97f6-d65a82f92523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:29.416106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24f1aa-e3a4-4f15-ac52-672f8e661d0f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:29.418485 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0a24f1aa-e3a4-4f15-ac52-672f8e661d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:29.425733 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1012/2028] 10.222.0.39 () {58 vars in 1018 bytes} [Thu Jul 28 22:29:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:29.453582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b01dc48-8168-4d16-97f6-d65a82f92523 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:29.453582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8b01dc48-8168-4d16-97f6-d65a82f92523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:29.461444 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:29.479543 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b01dc48-8168-4d16-97f6-d65a82f92523 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:29.486223 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9660b-86ee-4116-93e7-f97db5f1ac22 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe003cd7bfbe4b92b1d64d300c3e7d8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:29.486903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9660b-86ee-4116-93e7-f97db5f1ac22 service placement] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:29.487192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9660b-86ee-4116-93e7-f97db5f1ac22 service placement] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:29.487473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c9660b-86ee-4116-93e7-f97db5f1ac22 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:29.502056 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9660b-86ee-4116-93e7-f97db5f1ac22 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:29.504051 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c9660b-86ee-4116-93e7-f97db5f1ac22 service placement] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:29.518386 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1013/2029] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:29:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:29:29.531235 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a24f1aa-e3a4-4f15-ac52-672f8e661d0f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:29.536768 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7451b6f1-adff-42bc-96c1-0cd1537f1f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:29.537620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451b6f1-adff-42bc-96c1-0cd1537f1f21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:29.538424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451b6f1-adff-42bc-96c1-0cd1537f1f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:29.538797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7451b6f1-adff-42bc-96c1-0cd1537f1f21 admin admin] PATH_INFO: `/v3/users/4ed2359ca8cb4ee18984c14a5c3646b1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:29.540580 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451b6f1-adff-42bc-96c1-0cd1537f1f21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ed2359ca8cb4ee18984c14a5c3646b1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:29.542733 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7451b6f1-adff-42bc-96c1-0cd1537f1f21 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:29.599952 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7451b6f1-adff-42bc-96c1-0cd1537f1f21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ed2359ca8cb4ee18984c14a5c3646b1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:29.610568 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7451b6f1-adff-42bc-96c1-0cd1537f1f21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ed2359ca8cb4ee18984c14a5c3646b1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:29.618030 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7451b6f1-adff-42bc-96c1-0cd1537f1f21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ed2359ca8cb4ee18984c14a5c3646b1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:29.618691 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7451b6f1-adff-42bc-96c1-0cd1537f1f21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ed2359ca8cb4ee18984c14a5c3646b1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:29.631212 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7451b6f1-adff-42bc-96c1-0cd1537f1f21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ed2359ca8cb4ee18984c14a5c3646b1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:29.637419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1017/2030] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:29:29 2022] DELETE /identity/v3/users/4ed2359ca8cb4ee18984c14a5c3646b1 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:29.817228 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9660b-86ee-4116-93e7-f97db5f1ac22 service placement] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:29.830372 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c9660b-86ee-4116-93e7-f97db5f1ac22 service placement] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:29.906062 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ec7c71-833b-4bd1-872b-a9b87f89e820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:29.906723 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec7c71-833b-4bd1-872b-a9b87f89e820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:29.907119 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec7c71-833b-4bd1-872b-a9b87f89e820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:29.907591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ec7c71-833b-4bd1-872b-a9b87f89e820 admin admin] PATH_INFO: `/v3/projects/cb92681d2af8498cb7456a70354ccf39` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:29.908940 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec7c71-833b-4bd1-872b-a9b87f89e820 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb92681d2af8498cb7456a70354ccf39)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:29.910942 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ec7c71-833b-4bd1-872b-a9b87f89e820 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:29.964430 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:29.966057 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d6ec7c71-833b-4bd1-872b-a9b87f89e820 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb92681d2af8498cb7456a70354ccf39', 'request_id': 'req-d6ec7c71-833b-4bd1-872b-a9b87f89e820'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:29.974989 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d6ec7c71-833b-4bd1-872b-a9b87f89e820 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb92681d2af8498cb7456a70354ccf39', 'request_id': 'req-d6ec7c71-833b-4bd1-872b-a9b87f89e820'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:29.980794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d6ec7c71-833b-4bd1-872b-a9b87f89e820 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb92681d2af8498cb7456a70354ccf39', 'request_id': 'req-d6ec7c71-833b-4bd1-872b-a9b87f89e820'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:29.994525 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d6ec7c71-833b-4bd1-872b-a9b87f89e820 admin admin] The token cache is being invalidate because project cb92681d2af8498cb7456a70354ccf39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:29:29.994745 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d6ec7c71-833b-4bd1-872b-a9b87f89e820 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:29.995367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d6ec7c71-833b-4bd1-872b-a9b87f89e820 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:29.996891 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1014/2031] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:29:29 2022] DELETE /identity/v3/projects/cb92681d2af8498cb7456a70354ccf39 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:29:30.002666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ab34e-c044-4a42-b543-be01f8aed722 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:30.002959 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ab34e-c044-4a42-b543-be01f8aed722 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:30.003239 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ab34e-c044-4a42-b543-be01f8aed722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:30.023816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-469ab34e-c044-4a42-b543-be01f8aed722 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:30.025037 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-469ab34e-c044-4a42-b543-be01f8aed722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:30.085188 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1018/2032] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:30.093264 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce855d-6397-4d1d-b12b-c73bca77e2ed None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:30.093569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce855d-6397-4d1d-b12b-c73bca77e2ed None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:30.093845 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce855d-6397-4d1d-b12b-c73bca77e2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:30.116123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3dce855d-6397-4d1d-b12b-c73bca77e2ed None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:30.117348 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3dce855d-6397-4d1d-b12b-c73bca77e2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:30.129415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1015/2033] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:30.130617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ab34e-c044-4a42-b543-be01f8aed722 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:30.178337 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ab34e-c044-4a42-b543-be01f8aed722 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:30.184144 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e41edb-1cf0-415e-b8f3-88cd539d8b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:30.184943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e41edb-1cf0-415e-b8f3-88cd539d8b99 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:30.185395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e41edb-1cf0-415e-b8f3-88cd539d8b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:30.185709 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e41edb-1cf0-415e-b8f3-88cd539d8b99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:30.186415 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e41edb-1cf0-415e-b8f3-88cd539d8b99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:29:30.186721 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e41edb-1cf0-415e-b8f3-88cd539d8b99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:30.188358 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e41edb-1cf0-415e-b8f3-88cd539d8b99 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:30.203148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1019/2034] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:29:30.206606 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce855d-6397-4d1d-b12b-c73bca77e2ed None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:30.217033 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce855d-6397-4d1d-b12b-c73bca77e2ed None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:30.227510 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ba97c-0779-4c88-980d-d31c551184b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:30.228152 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ba97c-0779-4c88-980d-d31c551184b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:30.228434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ba97c-0779-4c88-980d-d31c551184b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:30.228709 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97ba97c-0779-4c88-980d-d31c551184b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:30.229379 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ba97c-0779-4c88-980d-d31c551184b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:30.231330 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97ba97c-0779-4c88-980d-d31c551184b3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:30.268089 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1016/2035] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:29:30 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:29:30.269251 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e41edb-1cf0-415e-b8f3-88cd539d8b99 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:30.280149 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e41edb-1cf0-415e-b8f3-88cd539d8b99 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:30.305749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:30.358802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72047ffb-6956-4b84-96aa-f982d16da914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:30.359516 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72047ffb-6956-4b84-96aa-f982d16da914 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:30.359852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72047ffb-6956-4b84-96aa-f982d16da914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:30.360212 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72047ffb-6956-4b84-96aa-f982d16da914 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:30.361469 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72047ffb-6956-4b84-96aa-f982d16da914 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:30.364524 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72047ffb-6956-4b84-96aa-f982d16da914 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:30.387482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1020/2036] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:29:30 2022] POST /identity/v3/users => generated 286 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:29:30.391731 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ba97c-0779-4c88-980d-d31c551184b3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:30.403489 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97ba97c-0779-4c88-980d-d31c551184b3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:30.408822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49a8a3-a78d-40c8-90d1-1fe41e8a3752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:30.409445 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49a8a3-a78d-40c8-90d1-1fe41e8a3752 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:30.409763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49a8a3-a78d-40c8-90d1-1fe41e8a3752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:30.410040 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49a8a3-a78d-40c8-90d1-1fe41e8a3752 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:30.410751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49a8a3-a78d-40c8-90d1-1fe41e8a3752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:30.412353 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49a8a3-a78d-40c8-90d1-1fe41e8a3752 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:30.422384 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1017/2037] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:29:30.426957 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72047ffb-6956-4b84-96aa-f982d16da914 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:30.436913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72047ffb-6956-4b84-96aa-f982d16da914 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:30.441688 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54056264-002c-47b8-a152-3e6d96fce02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:30.442303 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54056264-002c-47b8-a152-3e6d96fce02e admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:30.442590 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54056264-002c-47b8-a152-3e6d96fce02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:30.442870 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54056264-002c-47b8-a152-3e6d96fce02e admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:30.443569 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54056264-002c-47b8-a152-3e6d96fce02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:30.445170 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54056264-002c-47b8-a152-3e6d96fce02e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:30.455131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1021/2038] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:29:30.459128 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49a8a3-a78d-40c8-90d1-1fe41e8a3752 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:30.470218 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49a8a3-a78d-40c8-90d1-1fe41e8a3752 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:30.478132 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684c1d9-f9d1-4efc-b614-25fefbce5dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:30.478831 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684c1d9-f9d1-4efc-b614-25fefbce5dfd admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:30.479113 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684c1d9-f9d1-4efc-b614-25fefbce5dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:30.479392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684c1d9-f9d1-4efc-b614-25fefbce5dfd admin admin] PATH_INFO: `/v3/projects/7a1b7f30097644239d44b3a5ce0c17ab/users/4c07aa36c4ff47bcb75979cbfb06c6d4/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:30.495806 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684c1d9-f9d1-4efc-b614-25fefbce5dfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1b7f30097644239d44b3a5ce0c17ab, user_id=4c07aa36c4ff47bcb75979cbfb06c6d4, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:30.498543 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684c1d9-f9d1-4efc-b614-25fefbce5dfd admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:30.513089 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1018/2039] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:29:30 2022] PUT /identity/v3/projects/7a1b7f30097644239d44b3a5ce0c17ab/users/4c07aa36c4ff47bcb75979cbfb06c6d4/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:30.523538 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8432d-6138-4e90-b388-b6fa855a347a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:30.523831 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8432d-6138-4e90-b388-b6fa855a347a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:30.524113 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8432d-6138-4e90-b388-b6fa855a347a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:30.556332 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8fd8432d-6138-4e90-b388-b6fa855a347a None None] MFA Rules not processed for user `4c07aa36c4ff47bcb75979cbfb06c6d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:30.557706 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8fd8432d-6138-4e90-b388-b6fa855a347a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:30.629508 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1022/2040] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:30.634208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684c1d9-f9d1-4efc-b614-25fefbce5dfd admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:30.700594 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684c1d9-f9d1-4efc-b614-25fefbce5dfd admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:30.706190 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279477fc-acbc-4d8f-947a-82de8090e2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:30.706836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279477fc-acbc-4d8f-947a-82de8090e2c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:30.707122 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279477fc-acbc-4d8f-947a-82de8090e2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:30.707403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279477fc-acbc-4d8f-947a-82de8090e2c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:30.708152 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279477fc-acbc-4d8f-947a-82de8090e2c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:30.710179 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279477fc-acbc-4d8f-947a-82de8090e2c5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:30.739642 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1019/2041] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:29:30 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:29:30.743997 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8432d-6138-4e90-b388-b6fa855a347a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:30.756440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8432d-6138-4e90-b388-b6fa855a347a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:30.817964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d44fbd-4cdc-4ccf-b9c8-5b36583999c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:30.818516 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d44fbd-4cdc-4ccf-b9c8-5b36583999c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:30.818798 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d44fbd-4cdc-4ccf-b9c8-5b36583999c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:30.819072 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d44fbd-4cdc-4ccf-b9c8-5b36583999c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:30.819742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d44fbd-4cdc-4ccf-b9c8-5b36583999c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:30.821716 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d44fbd-4cdc-4ccf-b9c8-5b36583999c5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:30.848176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1023/2042] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:29:30 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:29:30.849373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279477fc-acbc-4d8f-947a-82de8090e2c5 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:30.861013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279477fc-acbc-4d8f-947a-82de8090e2c5 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:30.867199 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac1f36-ffd5-40e4-91dd-d12b2fbc9ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:30.867970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac1f36-ffd5-40e4-91dd-d12b2fbc9ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:30.868347 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac1f36-ffd5-40e4-91dd-d12b2fbc9ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:30.868732 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac1f36-ffd5-40e4-91dd-d12b2fbc9ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:30.869542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac1f36-ffd5-40e4-91dd-d12b2fbc9ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:30.871259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac1f36-ffd5-40e4-91dd-d12b2fbc9ab9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:30.884243 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1020/2043] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:29:30.888117 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d44fbd-4cdc-4ccf-b9c8-5b36583999c5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:30.911110 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d44fbd-4cdc-4ccf-b9c8-5b36583999c5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:30.916357 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223a0ea-c515-438b-b699-6a01d9122149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:30.917128 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223a0ea-c515-438b-b699-6a01d9122149 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:30.917484 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223a0ea-c515-438b-b699-6a01d9122149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:30.917951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3223a0ea-c515-438b-b699-6a01d9122149 admin admin] PATH_INFO: `/v3/projects/e21cd90479214083858125270dba05a7/users/564f035daadb4521be72b96bd750d719/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:30.934726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223a0ea-c515-438b-b699-6a01d9122149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e21cd90479214083858125270dba05a7, user_id=564f035daadb4521be72b96bd750d719, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:30.937456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3223a0ea-c515-438b-b699-6a01d9122149 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:30.948087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:30.958521 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1024/2044] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:29:30 2022] PUT /identity/v3/projects/e21cd90479214083858125270dba05a7/users/564f035daadb4521be72b96bd750d719/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:30.959552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac1f36-ffd5-40e4-91dd-d12b2fbc9ab9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:30.970706 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac1f36-ffd5-40e4-91dd-d12b2fbc9ab9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:31.018568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:31.036546 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d022e58c-045e-4001-b27b-cd7aa2ada959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:31.037300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022e58c-045e-4001-b27b-cd7aa2ada959 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:31.037760 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022e58c-045e-4001-b27b-cd7aa2ada959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:31.038042 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d022e58c-045e-4001-b27b-cd7aa2ada959 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:31.038779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022e58c-045e-4001-b27b-cd7aa2ada959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:31.040381 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d022e58c-045e-4001-b27b-cd7aa2ada959 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:31.052594 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1021/2045] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:29:30 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:29:31.056947 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223a0ea-c515-438b-b699-6a01d9122149 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:31.068180 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3223a0ea-c515-438b-b699-6a01d9122149 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:31.072837 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa0bd0-9b72-425e-a672-77a7861049e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:31.073509 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa0bd0-9b72-425e-a672-77a7861049e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:31.073826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa0bd0-9b72-425e-a672-77a7861049e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:31.074103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa0bd0-9b72-425e-a672-77a7861049e2 admin admin] PATH_INFO: `/v3/projects/e21cd90479214083858125270dba05a7/users/564f035daadb4521be72b96bd750d719/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:31.076062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa0bd0-9b72-425e-a672-77a7861049e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e21cd90479214083858125270dba05a7, user_id=564f035daadb4521be72b96bd750d719, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:31.078744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa0bd0-9b72-425e-a672-77a7861049e2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:31.092514 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1025/2046] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:29:31 2022] PUT /identity/v3/projects/e21cd90479214083858125270dba05a7/users/564f035daadb4521be72b96bd750d719/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:29:31.104564 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8714b7-1161-41cb-b756-a2adf8e8e935 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:31.104858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8714b7-1161-41cb-b756-a2adf8e8e935 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:31.105139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8714b7-1161-41cb-b756-a2adf8e8e935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:31.135784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ea8714b7-1161-41cb-b756-a2adf8e8e935 None None] MFA Rules not processed for user `564f035daadb4521be72b96bd750d719`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:31.137005 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ea8714b7-1161-41cb-b756-a2adf8e8e935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:31.232253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1022/2047] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:31 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:31.240740 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa0bd0-9b72-425e-a672-77a7861049e2 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:31.242802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0daa0bd0-9b72-425e-a672-77a7861049e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:31.308029 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa0bd0-9b72-425e-a672-77a7861049e2 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:31.312113 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf4db9-0730-4e50-bd3c-d67c7df4e3a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:31.312776 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf4db9-0730-4e50-bd3c-d67c7df4e3a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:31.313001 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf4db9-0730-4e50-bd3c-d67c7df4e3a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:31.313368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adf4db9-0730-4e50-bd3c-d67c7df4e3a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:31.323673 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf4db9-0730-4e50-bd3c-d67c7df4e3a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:31.325110 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adf4db9-0730-4e50-bd3c-d67c7df4e3a1 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:31.340020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1026/2048] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:29:31 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:29:31.449545 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8714b7-1161-41cb-b756-a2adf8e8e935 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:31.450986 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ea8714b7-1161-41cb-b756-a2adf8e8e935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:31.464022 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8714b7-1161-41cb-b756-a2adf8e8e935 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:31.468346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a22e47c-c1c9-4d80-a3f8-ac6bdae82521 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:31.468970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22e47c-c1c9-4d80-a3f8-ac6bdae82521 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:31.469256 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22e47c-c1c9-4d80-a3f8-ac6bdae82521 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:31.470281 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a22e47c-c1c9-4d80-a3f8-ac6bdae82521 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:31.530815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22e47c-c1c9-4d80-a3f8-ac6bdae82521 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:31.533792 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a22e47c-c1c9-4d80-a3f8-ac6bdae82521 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:31.544100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1023/2049] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:29:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:29:38.939681 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf4db9-0730-4e50-bd3c-d67c7df4e3a1 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:38.941768 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8adf4db9-0730-4e50-bd3c-d67c7df4e3a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:39.012593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adf4db9-0730-4e50-bd3c-d67c7df4e3a1 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:39.018326 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d256c-faa0-4d7e-8145-40400d505bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:39.018471 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d256c-faa0-4d7e-8145-40400d505bc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:39.018747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d256c-faa0-4d7e-8145-40400d505bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:39.019014 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690d256c-faa0-4d7e-8145-40400d505bc2 admin admin] PATH_INFO: `/v3/users/08bd37c3d9a04406b33c04be17deeb04` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:39.020187 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d256c-faa0-4d7e-8145-40400d505bc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08bd37c3d9a04406b33c04be17deeb04)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:39.022137 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690d256c-faa0-4d7e-8145-40400d505bc2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:39.077429 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-690d256c-faa0-4d7e-8145-40400d505bc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08bd37c3d9a04406b33c04be17deeb04'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:39.085258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-690d256c-faa0-4d7e-8145-40400d505bc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08bd37c3d9a04406b33c04be17deeb04'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:39.092412 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-690d256c-faa0-4d7e-8145-40400d505bc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08bd37c3d9a04406b33c04be17deeb04'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:39.093050 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-690d256c-faa0-4d7e-8145-40400d505bc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08bd37c3d9a04406b33c04be17deeb04'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:39.105791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-690d256c-faa0-4d7e-8145-40400d505bc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08bd37c3d9a04406b33c04be17deeb04'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:39.111854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1027/2050] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:29:38 2022] DELETE /identity/v3/users/08bd37c3d9a04406b33c04be17deeb04 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:39.116389 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616e0ff-4123-4d16-b41d-5f73a0bb2994 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:39.116876 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616e0ff-4123-4d16-b41d-5f73a0bb2994 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:39.116993 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a616e0ff-4123-4d16-b41d-5f73a0bb2994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:39.136966 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a616e0ff-4123-4d16-b41d-5f73a0bb2994 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:29:39.138698 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a616e0ff-4123-4d16-b41d-5f73a0bb2994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:39.199335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1024/2051] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:29:39.209571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d256c-faa0-4d7e-8145-40400d505bc2 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:39.210985 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-690d256c-faa0-4d7e-8145-40400d505bc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:39.221094 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:39.271539 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690d256c-faa0-4d7e-8145-40400d505bc2 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:39.275489 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4f971-46e0-4b11-89ad-5e41152ae1aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:39.276117 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4f971-46e0-4b11-89ad-5e41152ae1aa service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:39.276399 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4f971-46e0-4b11-89ad-5e41152ae1aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:39.276672 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d4f971-46e0-4b11-89ad-5e41152ae1aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:39.286798 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4f971-46e0-4b11-89ad-5e41152ae1aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:39.288237 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d4f971-46e0-4b11-89ad-5e41152ae1aa service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:39.297992 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1028/2052] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:29:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:29:39.590348 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616e0ff-4123-4d16-b41d-5f73a0bb2994 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:29:39.592652 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a616e0ff-4123-4d16-b41d-5f73a0bb2994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:29:39.608380 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a616e0ff-4123-4d16-b41d-5f73a0bb2994 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:29:39.613767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40a1f2-abec-4ead-b087-3be625ba3969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:29:39.614415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40a1f2-abec-4ead-b087-3be625ba3969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:29:39.614692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40a1f2-abec-4ead-b087-3be625ba3969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:29:39.614963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40a1f2-abec-4ead-b087-3be625ba3969 admin admin] PATH_INFO: `/v3/projects/4a853b6310d846a992dcab02c51a4057` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:29:39.616151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40a1f2-abec-4ead-b087-3be625ba3969 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a853b6310d846a992dcab02c51a4057)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:29:39.628695 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40a1f2-abec-4ead-b087-3be625ba3969 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:29:39.630630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:39.703611 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4b40a1f2-abec-4ead-b087-3be625ba3969 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a853b6310d846a992dcab02c51a4057', 'request_id': 'req-4b40a1f2-abec-4ead-b087-3be625ba3969'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:39.715300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4b40a1f2-abec-4ead-b087-3be625ba3969 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a853b6310d846a992dcab02c51a4057', 'request_id': 'req-4b40a1f2-abec-4ead-b087-3be625ba3969'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:39.721351 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4b40a1f2-abec-4ead-b087-3be625ba3969 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a853b6310d846a992dcab02c51a4057', 'request_id': 'req-4b40a1f2-abec-4ead-b087-3be625ba3969'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:39.733421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4b40a1f2-abec-4ead-b087-3be625ba3969 admin admin] The token cache is being invalidate because project 4a853b6310d846a992dcab02c51a4057 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:29:39.733740 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4b40a1f2-abec-4ead-b087-3be625ba3969 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:39.734382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4b40a1f2-abec-4ead-b087-3be625ba3969 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:29:39.735899 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1025/2053] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:29:39 2022] DELETE /identity/v3/projects/4a853b6310d846a992dcab02c51a4057 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:29:49.293965 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=23 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:29:49.730253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=24,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:02.797838 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4f971-46e0-4b11-89ad-5e41152ae1aa service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:02.799258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-36d4f971-46e0-4b11-89ad-5e41152ae1aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:02.813149 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d4f971-46e0-4b11-89ad-5e41152ae1aa service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:02.872621 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194f770-3c0e-4130-8f77-2bfaf490b9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:02.873275 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194f770-3c0e-4130-8f77-2bfaf490b9ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:02.873597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194f770-3c0e-4130-8f77-2bfaf490b9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:02.873878 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194f770-3c0e-4130-8f77-2bfaf490b9ca admin admin] PATH_INFO: `/v3/users/e904aa4dcc784227919619349c759472` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:02.875052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194f770-3c0e-4130-8f77-2bfaf490b9ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=e904aa4dcc784227919619349c759472)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:02.876974 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194f770-3c0e-4130-8f77-2bfaf490b9ca admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:02.924546 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7194f770-3c0e-4130-8f77-2bfaf490b9ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e904aa4dcc784227919619349c759472'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:02.933508 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7194f770-3c0e-4130-8f77-2bfaf490b9ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e904aa4dcc784227919619349c759472'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:02.941048 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7194f770-3c0e-4130-8f77-2bfaf490b9ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e904aa4dcc784227919619349c759472'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:02.942335 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7194f770-3c0e-4130-8f77-2bfaf490b9ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e904aa4dcc784227919619349c759472'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:02.954592 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7194f770-3c0e-4130-8f77-2bfaf490b9ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e904aa4dcc784227919619349c759472'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:02.960852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1029/2054] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:30:02 2022] DELETE /identity/v3/users/e904aa4dcc784227919619349c759472 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:30:02.965399 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e9b01-1fdf-4f18-9fdf-9d12a4f7c70e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:02.965903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e9b01-1fdf-4f18-9fdf-9d12a4f7c70e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:02.965903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919e9b01-1fdf-4f18-9fdf-9d12a4f7c70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:02.985751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-919e9b01-1fdf-4f18-9fdf-9d12a4f7c70e None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:02.987164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-919e9b01-1fdf-4f18-9fdf-9d12a4f7c70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:03.047315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1026/2055] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:30:03.058035 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194f770-3c0e-4130-8f77-2bfaf490b9ca admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:03.059453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7194f770-3c0e-4130-8f77-2bfaf490b9ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:03.125253 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194f770-3c0e-4130-8f77-2bfaf490b9ca admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:03.129820 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84db85-6188-417e-9b7d-e3b46e0d2148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:03.130452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84db85-6188-417e-9b7d-e3b46e0d2148 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:03.130740 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84db85-6188-417e-9b7d-e3b46e0d2148 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:03.131015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84db85-6188-417e-9b7d-e3b46e0d2148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:03.143932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84db85-6188-417e-9b7d-e3b46e0d2148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:03.146674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84db85-6188-417e-9b7d-e3b46e0d2148 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:03.162948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1030/2056] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:30:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:30:03.442499 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e9b01-1fdf-4f18-9fdf-9d12a4f7c70e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:03.444035 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-919e9b01-1fdf-4f18-9fdf-9d12a4f7c70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:03.459957 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919e9b01-1fdf-4f18-9fdf-9d12a4f7c70e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:03.464691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d436dc7c-fe08-406f-8b52-006921586fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:03.465560 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436dc7c-fe08-406f-8b52-006921586fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:03.465961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436dc7c-fe08-406f-8b52-006921586fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:03.466341 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d436dc7c-fe08-406f-8b52-006921586fa2 admin admin] PATH_INFO: `/v3/projects/9f1b46f6a60c4fac89d0f22260e3f8d6` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:03.467604 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436dc7c-fe08-406f-8b52-006921586fa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f1b46f6a60c4fac89d0f22260e3f8d6)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:03.469495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d436dc7c-fe08-406f-8b52-006921586fa2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:03.516441 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d436dc7c-fe08-406f-8b52-006921586fa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f1b46f6a60c4fac89d0f22260e3f8d6', 'request_id': 'req-d436dc7c-fe08-406f-8b52-006921586fa2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:03.523208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d436dc7c-fe08-406f-8b52-006921586fa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f1b46f6a60c4fac89d0f22260e3f8d6', 'request_id': 'req-d436dc7c-fe08-406f-8b52-006921586fa2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:03.528699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d436dc7c-fe08-406f-8b52-006921586fa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f1b46f6a60c4fac89d0f22260e3f8d6', 'request_id': 'req-d436dc7c-fe08-406f-8b52-006921586fa2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:03.541950 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d436dc7c-fe08-406f-8b52-006921586fa2 admin admin] The token cache is being invalidate because project 9f1b46f6a60c4fac89d0f22260e3f8d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:30:03.542239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d436dc7c-fe08-406f-8b52-006921586fa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:03.542876 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d436dc7c-fe08-406f-8b52-006921586fa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:03.544369 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1027/2057] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:30:03 2022] DELETE /identity/v3/projects/9f1b46f6a60c4fac89d0f22260e3f8d6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:30:03.549199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a400a-38cb-4dc4-a94a-c819bb70dbb2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:03.549485 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a400a-38cb-4dc4-a94a-c819bb70dbb2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:03.549799 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a400a-38cb-4dc4-a94a-c819bb70dbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:03.570226 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-417a400a-38cb-4dc4-a94a-c819bb70dbb2 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:03.571444 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-417a400a-38cb-4dc4-a94a-c819bb70dbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:03.588858 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:03.634466 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1031/2058] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:30:03.642313 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edb501-51f9-415b-b141-b9916af510f5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:03.642602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edb501-51f9-415b-b141-b9916af510f5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:03.642880 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97edb501-51f9-415b-b141-b9916af510f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:03.663344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-97edb501-51f9-415b-b141-b9916af510f5 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:03.664565 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-97edb501-51f9-415b-b141-b9916af510f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:03.672328 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1028/2059] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:30:03.676742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a400a-38cb-4dc4-a94a-c819bb70dbb2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:03.686187 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a400a-38cb-4dc4-a94a-c819bb70dbb2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:03.690652 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e854c-ab50-4a99-bbf2-39768167dcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:03.691252 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e854c-ab50-4a99-bbf2-39768167dcba admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:03.691529 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e854c-ab50-4a99-bbf2-39768167dcba admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:03.691799 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2e854c-ab50-4a99-bbf2-39768167dcba admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:03.692513 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e854c-ab50-4a99-bbf2-39768167dcba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:30:03.692819 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e854c-ab50-4a99-bbf2-39768167dcba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:03.694428 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2e854c-ab50-4a99-bbf2-39768167dcba admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:03.706434 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1032/2060] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:30:03.711162 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edb501-51f9-415b-b141-b9916af510f5 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:03.720392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97edb501-51f9-415b-b141-b9916af510f5 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:03.725828 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588368b-0796-4a19-a2af-9415ed98fc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:03.726429 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588368b-0796-4a19-a2af-9415ed98fc7b admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:03.726707 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588368b-0796-4a19-a2af-9415ed98fc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:03.726973 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3588368b-0796-4a19-a2af-9415ed98fc7b admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:03.727632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588368b-0796-4a19-a2af-9415ed98fc7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:03.729516 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3588368b-0796-4a19-a2af-9415ed98fc7b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:03.756918 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1029/2061] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:30:03 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:30:03.761350 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e854c-ab50-4a99-bbf2-39768167dcba admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:03.772120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2e854c-ab50-4a99-bbf2-39768167dcba admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:03.827406 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f578eb-d4e0-44d6-b84f-3efae23d6d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:03.828027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f578eb-d4e0-44d6-b84f-3efae23d6d3a admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:03.828315 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f578eb-d4e0-44d6-b84f-3efae23d6d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:03.828592 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f578eb-d4e0-44d6-b84f-3efae23d6d3a admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:03.829268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f578eb-d4e0-44d6-b84f-3efae23d6d3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:03.831215 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f578eb-d4e0-44d6-b84f-3efae23d6d3a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:03.853869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1033/2062] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:30:03 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:30:03.858026 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588368b-0796-4a19-a2af-9415ed98fc7b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:03.870368 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3588368b-0796-4a19-a2af-9415ed98fc7b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:03.875406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656b5ca-49dd-456a-a501-1896da14c7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:03.876022 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656b5ca-49dd-456a-a501-1896da14c7d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:03.876308 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656b5ca-49dd-456a-a501-1896da14c7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:03.876587 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5656b5ca-49dd-456a-a501-1896da14c7d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:03.877289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5656b5ca-49dd-456a-a501-1896da14c7d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:03.879079 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5656b5ca-49dd-456a-a501-1896da14c7d7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:03.894315 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1030/2063] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:30:03.895988 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f578eb-d4e0-44d6-b84f-3efae23d6d3a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:03.908039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f578eb-d4e0-44d6-b84f-3efae23d6d3a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:03.913512 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492106a-3bb2-44f8-9b51-d1b6b2330cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:03.914155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492106a-3bb2-44f8-9b51-d1b6b2330cee admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:03.914450 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492106a-3bb2-44f8-9b51-d1b6b2330cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:03.914705 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492106a-3bb2-44f8-9b51-d1b6b2330cee admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:03.915401 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492106a-3bb2-44f8-9b51-d1b6b2330cee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:03.917125 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492106a-3bb2-44f8-9b51-d1b6b2330cee admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:03.928193 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1034/2064] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:30:03.932602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656b5ca-49dd-456a-a501-1896da14c7d7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:03.943127 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5656b5ca-49dd-456a-a501-1896da14c7d7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:03.947886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861a5aa-1147-499f-89ba-4609c03c298f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:03.948538 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861a5aa-1147-499f-89ba-4609c03c298f admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:03.948818 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861a5aa-1147-499f-89ba-4609c03c298f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:03.949089 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861a5aa-1147-499f-89ba-4609c03c298f admin admin] PATH_INFO: `/v3/projects/b27af66b3c39453cbcd06fc06d51fce1/users/934fa56bb0474800bb45da0aa6a2088f/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:03.963562 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861a5aa-1147-499f-89ba-4609c03c298f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b27af66b3c39453cbcd06fc06d51fce1, user_id=934fa56bb0474800bb45da0aa6a2088f, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:03.966310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1861a5aa-1147-499f-89ba-4609c03c298f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:03.980623 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1031/2065] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:30:03 2022] PUT /identity/v3/projects/b27af66b3c39453cbcd06fc06d51fce1/users/934fa56bb0474800bb45da0aa6a2088f/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:30:03.991160 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c78a3-b312-4d76-b19a-c02689efdf70 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:03.991451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c78a3-b312-4d76-b19a-c02689efdf70 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:03.991757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c78a3-b312-4d76-b19a-c02689efdf70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:04.022042 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-922c78a3-b312-4d76-b19a-c02689efdf70 None None] MFA Rules not processed for user `934fa56bb0474800bb45da0aa6a2088f`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:04.023366 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-922c78a3-b312-4d76-b19a-c02689efdf70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:04.088707 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1035/2066] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:30:04.100854 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861a5aa-1147-499f-89ba-4609c03c298f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:04.102787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1861a5aa-1147-499f-89ba-4609c03c298f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:04.127181 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:04.174427 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861a5aa-1147-499f-89ba-4609c03c298f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:04.178747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390457b-b459-4da7-a723-f341c5b0ec50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:04.179591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390457b-b459-4da7-a723-f341c5b0ec50 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:04.179871 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390457b-b459-4da7-a723-f341c5b0ec50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:04.180160 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390457b-b459-4da7-a723-f341c5b0ec50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:04.189501 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390457b-b459-4da7-a723-f341c5b0ec50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:04.191040 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390457b-b459-4da7-a723-f341c5b0ec50 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:04.200633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1032/2067] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:30:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:30:04.228756 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c78a3-b312-4d76-b19a-c02689efdf70 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:04.249022 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:04.294019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c78a3-b312-4d76-b19a-c02689efdf70 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:04.298702 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7a53b-07ef-47bc-8cfb-b70944dd2a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:04.299343 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7a53b-07ef-47bc-8cfb-b70944dd2a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:04.299623 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7a53b-07ef-47bc-8cfb-b70944dd2a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:04.299892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7a53b-07ef-47bc-8cfb-b70944dd2a6a admin admin] PATH_INFO: `/v3/users/934fa56bb0474800bb45da0aa6a2088f` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:04.301077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7a53b-07ef-47bc-8cfb-b70944dd2a6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=934fa56bb0474800bb45da0aa6a2088f)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:04.304795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7a53b-07ef-47bc-8cfb-b70944dd2a6a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:04.353363 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5ed7a53b-07ef-47bc-8cfb-b70944dd2a6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '934fa56bb0474800bb45da0aa6a2088f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:04.361255 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5ed7a53b-07ef-47bc-8cfb-b70944dd2a6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '934fa56bb0474800bb45da0aa6a2088f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:04.369611 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5ed7a53b-07ef-47bc-8cfb-b70944dd2a6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '934fa56bb0474800bb45da0aa6a2088f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:04.370278 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5ed7a53b-07ef-47bc-8cfb-b70944dd2a6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '934fa56bb0474800bb45da0aa6a2088f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:04.383859 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5ed7a53b-07ef-47bc-8cfb-b70944dd2a6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '934fa56bb0474800bb45da0aa6a2088f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:04.390008 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1036/2068] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:30:04 2022] DELETE /identity/v3/users/934fa56bb0474800bb45da0aa6a2088f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:30:04.395041 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca84035-f5fe-4ec4-bdca-a69a5f1447fc None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:04.395539 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca84035-f5fe-4ec4-bdca-a69a5f1447fc None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:04.395917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca84035-f5fe-4ec4-bdca-a69a5f1447fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:04.416025 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-aca84035-f5fe-4ec4-bdca-a69a5f1447fc None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:04.422877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-aca84035-f5fe-4ec4-bdca-a69a5f1447fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:04.485760 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1033/2069] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:30:04.496457 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7a53b-07ef-47bc-8cfb-b70944dd2a6a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:04.497897 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5ed7a53b-07ef-47bc-8cfb-b70944dd2a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:04.560893 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7a53b-07ef-47bc-8cfb-b70944dd2a6a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:04.564325 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf1d799-3a42-4343-84d6-648f508a75c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:04.564946 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1d799-3a42-4343-84d6-648f508a75c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:04.565232 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1d799-3a42-4343-84d6-648f508a75c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:04.565646 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf1d799-3a42-4343-84d6-648f508a75c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:04.576542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1d799-3a42-4343-84d6-648f508a75c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:04.578033 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf1d799-3a42-4343-84d6-648f508a75c9 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:04.588465 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1037/2070] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:30:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:30:04.866420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca84035-f5fe-4ec4-bdca-a69a5f1447fc None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:04.868764 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-aca84035-f5fe-4ec4-bdca-a69a5f1447fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:04.886819 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca84035-f5fe-4ec4-bdca-a69a5f1447fc None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:04.894661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2de78b7-f467-40b9-be9f-fd93ee182562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:04.895844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de78b7-f467-40b9-be9f-fd93ee182562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:04.896363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de78b7-f467-40b9-be9f-fd93ee182562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:04.896882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2de78b7-f467-40b9-be9f-fd93ee182562 admin admin] PATH_INFO: `/v3/projects/b27af66b3c39453cbcd06fc06d51fce1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:04.898438 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de78b7-f467-40b9-be9f-fd93ee182562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b27af66b3c39453cbcd06fc06d51fce1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:04.900925 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2de78b7-f467-40b9-be9f-fd93ee182562 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:04.955540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f2de78b7-f467-40b9-be9f-fd93ee182562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b27af66b3c39453cbcd06fc06d51fce1', 'request_id': 'req-f2de78b7-f467-40b9-be9f-fd93ee182562'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:04.962463 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f2de78b7-f467-40b9-be9f-fd93ee182562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b27af66b3c39453cbcd06fc06d51fce1', 'request_id': 'req-f2de78b7-f467-40b9-be9f-fd93ee182562'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:04.967974 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f2de78b7-f467-40b9-be9f-fd93ee182562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b27af66b3c39453cbcd06fc06d51fce1', 'request_id': 'req-f2de78b7-f467-40b9-be9f-fd93ee182562'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:04.982180 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f2de78b7-f467-40b9-be9f-fd93ee182562 admin admin] The token cache is being invalidate because project b27af66b3c39453cbcd06fc06d51fce1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:30:04.982468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f2de78b7-f467-40b9-be9f-fd93ee182562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:04.983085 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f2de78b7-f467-40b9-be9f-fd93ee182562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:04.984600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1034/2071] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:30:04 2022] DELETE /identity/v3/projects/b27af66b3c39453cbcd06fc06d51fce1 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:30:04.990606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9237e-5f16-40cd-83fe-2477e06ea04a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:04.991059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9237e-5f16-40cd-83fe-2477e06ea04a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:04.991150 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d9237e-5f16-40cd-83fe-2477e06ea04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:05.011461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e3d9237e-5f16-40cd-83fe-2477e06ea04a None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:05.012670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e3d9237e-5f16-40cd-83fe-2477e06ea04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:05.045156 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:05.072719 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1038/2072] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:30:05.081484 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963e989-4c7e-419e-9c04-6176dc454409 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:05.081800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963e989-4c7e-419e-9c04-6176dc454409 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:05.082102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9963e989-4c7e-419e-9c04-6176dc454409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:05.102555 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9963e989-4c7e-419e-9c04-6176dc454409 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:05.103748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9963e989-4c7e-419e-9c04-6176dc454409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:05.111665 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1035/2073] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:30:05.116725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9237e-5f16-40cd-83fe-2477e06ea04a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:05.126304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d9237e-5f16-40cd-83fe-2477e06ea04a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:05.131925 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b8550-ce39-474d-aaba-8b2aefdeec72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:05.132782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b8550-ce39-474d-aaba-8b2aefdeec72 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:05.133065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b8550-ce39-474d-aaba-8b2aefdeec72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:05.133341 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b8550-ce39-474d-aaba-8b2aefdeec72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:05.134081 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b8550-ce39-474d-aaba-8b2aefdeec72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:30:05.134388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b8550-ce39-474d-aaba-8b2aefdeec72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:05.135983 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b8550-ce39-474d-aaba-8b2aefdeec72 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:05.147887 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1039/2074] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:30:05.151881 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963e989-4c7e-419e-9c04-6176dc454409 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:05.161422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9963e989-4c7e-419e-9c04-6176dc454409 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:05.166103 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bceaa5e-7a33-4252-80d4-5b51d41c8260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:05.166704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceaa5e-7a33-4252-80d4-5b51d41c8260 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:05.166979 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceaa5e-7a33-4252-80d4-5b51d41c8260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:05.167251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bceaa5e-7a33-4252-80d4-5b51d41c8260 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:05.167915 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bceaa5e-7a33-4252-80d4-5b51d41c8260 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:05.169835 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bceaa5e-7a33-4252-80d4-5b51d41c8260 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:05.197475 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1036/2075] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:30:05 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:30:05.201563 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b8550-ce39-474d-aaba-8b2aefdeec72 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:05.211850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b8550-ce39-474d-aaba-8b2aefdeec72 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:05.269790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70568200-b2c6-4ea1-8995-89426769e6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:05.270450 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70568200-b2c6-4ea1-8995-89426769e6da admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:05.270858 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70568200-b2c6-4ea1-8995-89426769e6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:05.270995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70568200-b2c6-4ea1-8995-89426769e6da admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:05.271672 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70568200-b2c6-4ea1-8995-89426769e6da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:05.273656 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70568200-b2c6-4ea1-8995-89426769e6da admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:05.299597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1040/2076] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:30:05 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:30:05.300739 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bceaa5e-7a33-4252-80d4-5b51d41c8260 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:05.314777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bceaa5e-7a33-4252-80d4-5b51d41c8260 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:05.319572 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497987fc-5256-44ea-93cf-1f79a26619cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:05.320182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497987fc-5256-44ea-93cf-1f79a26619cd admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:05.320472 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497987fc-5256-44ea-93cf-1f79a26619cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:05.320742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497987fc-5256-44ea-93cf-1f79a26619cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:05.321444 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497987fc-5256-44ea-93cf-1f79a26619cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:05.323063 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497987fc-5256-44ea-93cf-1f79a26619cd admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:05.337391 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1037/2077] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:30:05.338687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70568200-b2c6-4ea1-8995-89426769e6da admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:05.352483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70568200-b2c6-4ea1-8995-89426769e6da admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:05.357335 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c5e9e2-7487-43ef-88d2-2968a6564f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:05.357930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5e9e2-7487-43ef-88d2-2968a6564f93 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:05.358211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5e9e2-7487-43ef-88d2-2968a6564f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:05.358486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c5e9e2-7487-43ef-88d2-2968a6564f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:05.359189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5e9e2-7487-43ef-88d2-2968a6564f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:05.360791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c5e9e2-7487-43ef-88d2-2968a6564f93 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:05.375993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1041/2078] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:30:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:30:05.377256 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497987fc-5256-44ea-93cf-1f79a26619cd admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:05.387820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497987fc-5256-44ea-93cf-1f79a26619cd admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:05.392686 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5d9b8-c5ba-478d-8867-99fe0fbc00ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:05.393405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5d9b8-c5ba-478d-8867-99fe0fbc00ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:05.393684 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5d9b8-c5ba-478d-8867-99fe0fbc00ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:05.393977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a5d9b8-c5ba-478d-8867-99fe0fbc00ce admin admin] PATH_INFO: `/v3/projects/2406f01f7b3b4941a0bcd97a68d3d510/users/866795813bc545b5a9293ca714095679/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:05.396505 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:05.413810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5d9b8-c5ba-478d-8867-99fe0fbc00ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=2406f01f7b3b4941a0bcd97a68d3d510, user_id=866795813bc545b5a9293ca714095679, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:05.416491 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a5d9b8-c5ba-478d-8867-99fe0fbc00ce admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:05.432251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1038/2079] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:30:05 2022] PUT /identity/v3/projects/2406f01f7b3b4941a0bcd97a68d3d510/users/866795813bc545b5a9293ca714095679/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:30:05.437332 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139e069-0368-49f2-8885-c27fd79f74ff None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:05.437700 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139e069-0368-49f2-8885-c27fd79f74ff None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:05.437986 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139e069-0368-49f2-8885-c27fd79f74ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:05.460387 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8139e069-0368-49f2-8885-c27fd79f74ff None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:05.462055 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8139e069-0368-49f2-8885-c27fd79f74ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:05.531994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1042/2080] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:30:05.542668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5d9b8-c5ba-478d-8867-99fe0fbc00ce admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:05.545200 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f2a5d9b8-c5ba-478d-8867-99fe0fbc00ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:05.616822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a5d9b8-c5ba-478d-8867-99fe0fbc00ce admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:05.620438 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fc0bb-3f05-43a5-b5c1-902f6bf2cc27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:05.621089 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fc0bb-3f05-43a5-b5c1-902f6bf2cc27 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:05.621375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fc0bb-3f05-43a5-b5c1-902f6bf2cc27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:05.621675 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fc0bb-3f05-43a5-b5c1-902f6bf2cc27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:05.631343 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fc0bb-3f05-43a5-b5c1-902f6bf2cc27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:05.633131 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fc0bb-3f05-43a5-b5c1-902f6bf2cc27 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:05.643546 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1039/2081] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:30:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:30:10.908829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37df0b4-40ee-471d-98a9-10a41eba8114 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:10.909319 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37df0b4-40ee-471d-98a9-10a41eba8114 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:10.909397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37df0b4-40ee-471d-98a9-10a41eba8114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:10.959681 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a37df0b4-40ee-471d-98a9-10a41eba8114 None None] MFA Rules not processed for user `866795813bc545b5a9293ca714095679`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:10.960919 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a37df0b4-40ee-471d-98a9-10a41eba8114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:11.001405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:11.038342 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1043/2082] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:30:11.042874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fc0bb-3f05-43a5-b5c1-902f6bf2cc27 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:11.053289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fc0bb-3f05-43a5-b5c1-902f6bf2cc27 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:11.058059 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfbeba6-3fdb-4d4f-9c7f-8b0eeffb5c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:11.058706 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbeba6-3fdb-4d4f-9c7f-8b0eeffb5c23 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:11.058983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbeba6-3fdb-4d4f-9c7f-8b0eeffb5c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:11.059250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfbeba6-3fdb-4d4f-9c7f-8b0eeffb5c23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:11.059918 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfbeba6-3fdb-4d4f-9c7f-8b0eeffb5c23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:11.061884 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfbeba6-3fdb-4d4f-9c7f-8b0eeffb5c23 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:11.091044 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1040/2083] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:30:11 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:30:11.095266 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37df0b4-40ee-471d-98a9-10a41eba8114 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:11.106639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37df0b4-40ee-471d-98a9-10a41eba8114 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:11.174220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420c630-a484-4bfa-b0f0-df0fbc7c2605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:11.174855 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420c630-a484-4bfa-b0f0-df0fbc7c2605 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:11.175138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420c630-a484-4bfa-b0f0-df0fbc7c2605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:11.175413 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420c630-a484-4bfa-b0f0-df0fbc7c2605 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:11.176609 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420c630-a484-4bfa-b0f0-df0fbc7c2605 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:11.178562 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1420c630-a484-4bfa-b0f0-df0fbc7c2605 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:11.202449 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1044/2084] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:30:11 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:30:11.206514 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfbeba6-3fdb-4d4f-9c7f-8b0eeffb5c23 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:11.218518 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfbeba6-3fdb-4d4f-9c7f-8b0eeffb5c23 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:11.224930 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c70b44-f769-482b-bc43-0deebda23e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:11.225679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c70b44-f769-482b-bc43-0deebda23e48 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:11.225970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c70b44-f769-482b-bc43-0deebda23e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:11.226249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c70b44-f769-482b-bc43-0deebda23e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:11.227025 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c70b44-f769-482b-bc43-0deebda23e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:11.228828 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c70b44-f769-482b-bc43-0deebda23e48 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:11.241403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1041/2085] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:30:11.245743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420c630-a484-4bfa-b0f0-df0fbc7c2605 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:11.256208 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1420c630-a484-4bfa-b0f0-df0fbc7c2605 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:11.261481 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66605464-8123-49be-968a-d967c65ef7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:11.262194 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66605464-8123-49be-968a-d967c65ef7c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:11.262635 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66605464-8123-49be-968a-d967c65ef7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:11.263111 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66605464-8123-49be-968a-d967c65ef7c7 admin admin] PATH_INFO: `/v3/projects/80d3cde38cd9481ab0dc9d3836de3359/users/a094696b3532489c8b585fed82aca471/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:11.283240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66605464-8123-49be-968a-d967c65ef7c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80d3cde38cd9481ab0dc9d3836de3359, user_id=a094696b3532489c8b585fed82aca471, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:11.287600 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66605464-8123-49be-968a-d967c65ef7c7 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:11.305766 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1045/2086] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:30:11 2022] PUT /identity/v3/projects/80d3cde38cd9481ab0dc9d3836de3359/users/a094696b3532489c8b585fed82aca471/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:30:11.309799 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c70b44-f769-482b-bc43-0deebda23e48 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:11.321803 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c70b44-f769-482b-bc43-0deebda23e48 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:11.386264 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a7f84-1deb-4910-bc07-54e11c0acf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:11.386882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a7f84-1deb-4910-bc07-54e11c0acf90 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:11.387167 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a7f84-1deb-4910-bc07-54e11c0acf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:11.387442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a7f84-1deb-4910-bc07-54e11c0acf90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:11.388137 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a7f84-1deb-4910-bc07-54e11c0acf90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:11.389800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a7f84-1deb-4910-bc07-54e11c0acf90 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:11.399719 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1042/2087] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:30:11 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:30:11.403836 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66605464-8123-49be-968a-d967c65ef7c7 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:11.414866 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66605464-8123-49be-968a-d967c65ef7c7 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:11.419468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be853d3-54ee-4a51-aa36-122ece990cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:11.420125 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be853d3-54ee-4a51-aa36-122ece990cfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:11.420407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be853d3-54ee-4a51-aa36-122ece990cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:11.420679 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be853d3-54ee-4a51-aa36-122ece990cfa admin admin] PATH_INFO: `/v3/projects/80d3cde38cd9481ab0dc9d3836de3359/users/a094696b3532489c8b585fed82aca471/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:11.422693 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be853d3-54ee-4a51-aa36-122ece990cfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=80d3cde38cd9481ab0dc9d3836de3359, user_id=a094696b3532489c8b585fed82aca471, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:11.425423 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be853d3-54ee-4a51-aa36-122ece990cfa admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:11.442216 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1046/2088] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:30:11 2022] PUT /identity/v3/projects/80d3cde38cd9481ab0dc9d3836de3359/users/a094696b3532489c8b585fed82aca471/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:30:16.921231 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a7f84-1deb-4910-bc07-54e11c0acf90 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:16.923916 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e00a7f84-1deb-4910-bc07-54e11c0acf90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:16.938420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a7f84-1deb-4910-bc07-54e11c0acf90 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:17.006805 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:17.025610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c099bc4-ab93-4f1c-9953-5ba1230b3ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:17.026625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c099bc4-ab93-4f1c-9953-5ba1230b3ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:17.027081 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c099bc4-ab93-4f1c-9953-5ba1230b3ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:17.027502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c099bc4-ab93-4f1c-9953-5ba1230b3ffd admin admin] PATH_INFO: `/v3/users/4c07aa36c4ff47bcb75979cbfb06c6d4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:17.029340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c099bc4-ab93-4f1c-9953-5ba1230b3ffd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c07aa36c4ff47bcb75979cbfb06c6d4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:17.032082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c099bc4-ab93-4f1c-9953-5ba1230b3ffd admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:17.104690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0c099bc4-ab93-4f1c-9953-5ba1230b3ffd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c07aa36c4ff47bcb75979cbfb06c6d4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:17.116288 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0c099bc4-ab93-4f1c-9953-5ba1230b3ffd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c07aa36c4ff47bcb75979cbfb06c6d4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:17.117518 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0c099bc4-ab93-4f1c-9953-5ba1230b3ffd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c07aa36c4ff47bcb75979cbfb06c6d4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:17.118490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0c099bc4-ab93-4f1c-9953-5ba1230b3ffd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c07aa36c4ff47bcb75979cbfb06c6d4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:17.140068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0c099bc4-ab93-4f1c-9953-5ba1230b3ffd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c07aa36c4ff47bcb75979cbfb06c6d4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:17.165783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1043/2089] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:30:16 2022] DELETE /identity/v3/users/4c07aa36c4ff47bcb75979cbfb06c6d4 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:30:17.171509 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0321b-cef7-4399-8e99-99cffa0ad560 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:17.171989 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0321b-cef7-4399-8e99-99cffa0ad560 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:17.172234 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0321b-cef7-4399-8e99-99cffa0ad560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:17.197240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c8a0321b-cef7-4399-8e99-99cffa0ad560 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:17.198786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c8a0321b-cef7-4399-8e99-99cffa0ad560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:17.283002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1047/2090] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:30:17.293054 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c099bc4-ab93-4f1c-9953-5ba1230b3ffd admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:17.298210 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0c099bc4-ab93-4f1c-9953-5ba1230b3ffd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:17.385805 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c099bc4-ab93-4f1c-9953-5ba1230b3ffd admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:17.389897 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc8943-fe5e-4cf2-89b4-49ab85caa2e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:17.390170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc8943-fe5e-4cf2-89b4-49ab85caa2e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:17.390402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc8943-fe5e-4cf2-89b4-49ab85caa2e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:17.390923 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc8943-fe5e-4cf2-89b4-49ab85caa2e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:17.404912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc8943-fe5e-4cf2-89b4-49ab85caa2e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:17.404912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc8943-fe5e-4cf2-89b4-49ab85caa2e7 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:17.417209 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1044/2091] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:30:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:30:18.003517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0321b-cef7-4399-8e99-99cffa0ad560 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:18.005950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c8a0321b-cef7-4399-8e99-99cffa0ad560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:18.023493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0321b-cef7-4399-8e99-99cffa0ad560 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:18.044598 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaee7d9-d6d4-43d4-9bea-8c013e0995a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:18.045491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaee7d9-d6d4-43d4-9bea-8c013e0995a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:18.045840 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaee7d9-d6d4-43d4-9bea-8c013e0995a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:18.046104 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaee7d9-d6d4-43d4-9bea-8c013e0995a9 admin admin] PATH_INFO: `/v3/projects/7a1b7f30097644239d44b3a5ce0c17ab` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:18.050483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaee7d9-d6d4-43d4-9bea-8c013e0995a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a1b7f30097644239d44b3a5ce0c17ab)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:18.053243 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaee7d9-d6d4-43d4-9bea-8c013e0995a9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:18.102871 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:18.132942 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ad858-c0ba-418b-a161-99a1a4301e75 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:18.133448 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ad858-c0ba-418b-a161-99a1a4301e75 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:18.133635 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ad858-c0ba-418b-a161-99a1a4301e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:18.162166 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-beaee7d9-d6d4-43d4-9bea-8c013e0995a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a1b7f30097644239d44b3a5ce0c17ab', 'request_id': 'req-beaee7d9-d6d4-43d4-9bea-8c013e0995a9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:18.167787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-bd8ad858-c0ba-418b-a161-99a1a4301e75 None None] MFA Rules not processed for user `a094696b3532489c8b585fed82aca471`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:18.169055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bd8ad858-c0ba-418b-a161-99a1a4301e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:18.180650 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-beaee7d9-d6d4-43d4-9bea-8c013e0995a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a1b7f30097644239d44b3a5ce0c17ab', 'request_id': 'req-beaee7d9-d6d4-43d4-9bea-8c013e0995a9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:18.187422 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-beaee7d9-d6d4-43d4-9bea-8c013e0995a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a1b7f30097644239d44b3a5ce0c17ab', 'request_id': 'req-beaee7d9-d6d4-43d4-9bea-8c013e0995a9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:18.203043 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-beaee7d9-d6d4-43d4-9bea-8c013e0995a9 admin admin] The token cache is being invalidate because project 7a1b7f30097644239d44b3a5ce0c17ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:30:18.203311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-beaee7d9-d6d4-43d4-9bea-8c013e0995a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:18.204399 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-beaee7d9-d6d4-43d4-9bea-8c013e0995a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:18.209169 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1048/2092] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:30:18 2022] DELETE /identity/v3/projects/7a1b7f30097644239d44b3a5ce0c17ab => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:30:18.214124 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaee7d9-d6d4-43d4-9bea-8c013e0995a9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:18.216626 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-beaee7d9-d6d4-43d4-9bea-8c013e0995a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:18.260585 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:18.288108 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1045/2093] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:30:18.316068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ad858-c0ba-418b-a161-99a1a4301e75 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:18.319088 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bd8ad858-c0ba-418b-a161-99a1a4301e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:18.332703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaee7d9-d6d4-43d4-9bea-8c013e0995a9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:18.350796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe738e7f-d5d0-40ce-8ec9-75cebd0b3048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:18.351895 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe738e7f-d5d0-40ce-8ec9-75cebd0b3048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:18.352194 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe738e7f-d5d0-40ce-8ec9-75cebd0b3048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:18.352459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe738e7f-d5d0-40ce-8ec9-75cebd0b3048 admin admin] PATH_INFO: `/v3/users/564f035daadb4521be72b96bd750d719` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:18.353754 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe738e7f-d5d0-40ce-8ec9-75cebd0b3048 admin admin] RBAC: Authorizing `identity:delete_user(user_id=564f035daadb4521be72b96bd750d719)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:18.355803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe738e7f-d5d0-40ce-8ec9-75cebd0b3048 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:18.440656 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fe738e7f-d5d0-40ce-8ec9-75cebd0b3048 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '564f035daadb4521be72b96bd750d719'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:18.450776 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fe738e7f-d5d0-40ce-8ec9-75cebd0b3048 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '564f035daadb4521be72b96bd750d719'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:18.461035 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fe738e7f-d5d0-40ce-8ec9-75cebd0b3048 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '564f035daadb4521be72b96bd750d719'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:18.461614 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ad858-c0ba-418b-a161-99a1a4301e75 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:18.462116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fe738e7f-d5d0-40ce-8ec9-75cebd0b3048 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '564f035daadb4521be72b96bd750d719'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:18.475883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b9ff55-626e-46b2-9e82-5ed72f3e98d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:18.476526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9ff55-626e-46b2-9e82-5ed72f3e98d8 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:18.476805 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9ff55-626e-46b2-9e82-5ed72f3e98d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:18.477086 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b9ff55-626e-46b2-9e82-5ed72f3e98d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:18.481214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-01b9ff55-626e-46b2-9e82-5ed72f3e98d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:18.485924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fe738e7f-d5d0-40ce-8ec9-75cebd0b3048 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '564f035daadb4521be72b96bd750d719'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:18.493443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1049/2094] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:30:18 2022] DELETE /identity/v3/users/564f035daadb4521be72b96bd750d719 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:30:18.555939 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b9ff55-626e-46b2-9e82-5ed72f3e98d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:18.557636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b9ff55-626e-46b2-9e82-5ed72f3e98d8 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:18.570062 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1046/2095] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:30:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:30:19.083481 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe738e7f-d5d0-40ce-8ec9-75cebd0b3048 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:19.085713 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fe738e7f-d5d0-40ce-8ec9-75cebd0b3048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:19.153560 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:19.220551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe738e7f-d5d0-40ce-8ec9-75cebd0b3048 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:19.227483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4321cf00-ed6b-48fe-a249-76eeafd312f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:19.228179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321cf00-ed6b-48fe-a249-76eeafd312f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:19.228462 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321cf00-ed6b-48fe-a249-76eeafd312f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:19.228735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4321cf00-ed6b-48fe-a249-76eeafd312f1 admin admin] PATH_INFO: `/v3/projects/e21cd90479214083858125270dba05a7` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:19.229994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321cf00-ed6b-48fe-a249-76eeafd312f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e21cd90479214083858125270dba05a7)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:19.232642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4321cf00-ed6b-48fe-a249-76eeafd312f1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:19.300095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4321cf00-ed6b-48fe-a249-76eeafd312f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e21cd90479214083858125270dba05a7', 'request_id': 'req-4321cf00-ed6b-48fe-a249-76eeafd312f1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:19.310182 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4321cf00-ed6b-48fe-a249-76eeafd312f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e21cd90479214083858125270dba05a7', 'request_id': 'req-4321cf00-ed6b-48fe-a249-76eeafd312f1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:19.318331 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4321cf00-ed6b-48fe-a249-76eeafd312f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e21cd90479214083858125270dba05a7', 'request_id': 'req-4321cf00-ed6b-48fe-a249-76eeafd312f1'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:19.338930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4321cf00-ed6b-48fe-a249-76eeafd312f1 admin admin] The token cache is being invalidate because project e21cd90479214083858125270dba05a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:30:19.339228 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4321cf00-ed6b-48fe-a249-76eeafd312f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:19.339954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4321cf00-ed6b-48fe-a249-76eeafd312f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:19.341723 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1050/2096] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:30:19 2022] DELETE /identity/v3/projects/e21cd90479214083858125270dba05a7 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:30:19.346493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b212ecf-ba3f-487d-9f4a-80720fe7f18f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:19.346976 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b212ecf-ba3f-487d-9f4a-80720fe7f18f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:19.347076 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b212ecf-ba3f-487d-9f4a-80720fe7f18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:19.368808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3b212ecf-ba3f-487d-9f4a-80720fe7f18f None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:19.370058 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3b212ecf-ba3f-487d-9f4a-80720fe7f18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:19.440982 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1047/2097] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:30:19.449486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e0838-5bea-41b8-a74c-63996da6027b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:19.449830 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e0838-5bea-41b8-a74c-63996da6027b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:19.450237 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e0838-5bea-41b8-a74c-63996da6027b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:19.480799 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-908e0838-5bea-41b8-a74c-63996da6027b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:19.482104 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-908e0838-5bea-41b8-a74c-63996da6027b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:19.491256 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1051/2098] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:30:19.495630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b212ecf-ba3f-487d-9f4a-80720fe7f18f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:19.515863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b212ecf-ba3f-487d-9f4a-80720fe7f18f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:19.528646 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e573384-d57d-47b6-9783-c8ff1148e395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:19.529727 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e573384-d57d-47b6-9783-c8ff1148e395 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:19.530220 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e573384-d57d-47b6-9783-c8ff1148e395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:19.531415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e573384-d57d-47b6-9783-c8ff1148e395 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:19.533088 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e573384-d57d-47b6-9783-c8ff1148e395 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:30:19.533682 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e573384-d57d-47b6-9783-c8ff1148e395 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:19.536168 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e573384-d57d-47b6-9783-c8ff1148e395 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:19.553774 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1048/2099] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:30:19.554845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e0838-5bea-41b8-a74c-63996da6027b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:19.571610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e0838-5bea-41b8-a74c-63996da6027b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:19.594750 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe0743-4e0d-40ef-a8d8-9174990a8404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:19.595942 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe0743-4e0d-40ef-a8d8-9174990a8404 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:19.596311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe0743-4e0d-40ef-a8d8-9174990a8404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:19.597083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbe0743-4e0d-40ef-a8d8-9174990a8404 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:19.598817 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe0743-4e0d-40ef-a8d8-9174990a8404 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:19.601970 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbe0743-4e0d-40ef-a8d8-9174990a8404 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:19.685245 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1052/2100] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:30:19 2022] POST /identity/v3/projects => generated 379 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:30:19.689761 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e573384-d57d-47b6-9783-c8ff1148e395 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:19.701846 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e573384-d57d-47b6-9783-c8ff1148e395 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:19.739648 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:19.812406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4683567-0790-49ee-8567-0019f776ebb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:19.813045 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4683567-0790-49ee-8567-0019f776ebb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:19.813326 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4683567-0790-49ee-8567-0019f776ebb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:19.813775 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4683567-0790-49ee-8567-0019f776ebb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:19.814318 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4683567-0790-49ee-8567-0019f776ebb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:19.817212 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4683567-0790-49ee-8567-0019f776ebb4 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:19.888627 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1049/2101] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:30:19 2022] POST /identity/v3/users => generated 290 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:30:19.889236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe0743-4e0d-40ef-a8d8-9174990a8404 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:19.910945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbe0743-4e0d-40ef-a8d8-9174990a8404 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:19.921856 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d50894-43c6-4ee0-8003-a993455ac7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:19.948171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d50894-43c6-4ee0-8003-a993455ac7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:19.948486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d50894-43c6-4ee0-8003-a993455ac7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:19.948769 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d50894-43c6-4ee0-8003-a993455ac7f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:19.949506 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d50894-43c6-4ee0-8003-a993455ac7f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:19.951931 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d50894-43c6-4ee0-8003-a993455ac7f7 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:19.987848 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1053/2102] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:30:19.992045 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4683567-0790-49ee-8567-0019f776ebb4 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:20.020744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4683567-0790-49ee-8567-0019f776ebb4 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:20.052559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9e0b6-d0e9-41f3-b197-13691c3229dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:20.053704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9e0b6-d0e9-41f3-b197-13691c3229dc admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:20.054133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9e0b6-d0e9-41f3-b197-13691c3229dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:20.054562 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9e0b6-d0e9-41f3-b197-13691c3229dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:20.056973 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a9e0b6-d0e9-41f3-b197-13691c3229dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:20.059762 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a9e0b6-d0e9-41f3-b197-13691c3229dc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:20.112693 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1050/2103] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:30:20.114891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d50894-43c6-4ee0-8003-a993455ac7f7 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:20.168562 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d50894-43c6-4ee0-8003-a993455ac7f7 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:20.187751 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b8aca-e5b9-44a8-9a9b-1bd82441b74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:20.188925 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b8aca-e5b9-44a8-9a9b-1bd82441b74c admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:20.190478 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b8aca-e5b9-44a8-9a9b-1bd82441b74c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:20.191059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b8aca-e5b9-44a8-9a9b-1bd82441b74c admin admin] PATH_INFO: `/v3/projects/7a284e8caf29431cbeddecbb71d03594/users/756ba921e64d4a97a94b77b68940b0fe/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:20.197688 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d65490-5f8b-4a5b-8ab3-f277aad76fc8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:20.198076 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d65490-5f8b-4a5b-8ab3-f277aad76fc8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:20.198467 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d65490-5f8b-4a5b-8ab3-f277aad76fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:20.211636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b8aca-e5b9-44a8-9a9b-1bd82441b74c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a284e8caf29431cbeddecbb71d03594, user_id=756ba921e64d4a97a94b77b68940b0fe, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:20.214817 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b8aca-e5b9-44a8-9a9b-1bd82441b74c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:20.228781 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-06d65490-5f8b-4a5b-8ab3-f277aad76fc8 None None] MFA Rules not processed for user `866795813bc545b5a9293ca714095679`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:20.230708 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-06d65490-5f8b-4a5b-8ab3-f277aad76fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:20.237035 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1054/2104] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:30:20 2022] PUT /identity/v3/projects/7a284e8caf29431cbeddecbb71d03594/users/756ba921e64d4a97a94b77b68940b0fe/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:30:20.251264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1061ca-accc-497f-bcb1-9fb0a7401efa None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:20.251556 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1061ca-accc-497f-bcb1-9fb0a7401efa None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:20.252012 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1061ca-accc-497f-bcb1-9fb0a7401efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:20.301804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9a1061ca-accc-497f-bcb1-9fb0a7401efa None None] MFA Rules not processed for user `756ba921e64d4a97a94b77b68940b0fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:20.303186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9a1061ca-accc-497f-bcb1-9fb0a7401efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:20.341678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1051/2105] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:30:20.372129 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:20.413232 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1055/2106] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:30:20.426895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d65490-5f8b-4a5b-8ab3-f277aad76fc8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:20.497334 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d65490-5f8b-4a5b-8ab3-f277aad76fc8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:20.501906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39453d28-166e-4cb9-8d91-d3a0f2f74881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:20.502480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39453d28-166e-4cb9-8d91-d3a0f2f74881 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:20.502761 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39453d28-166e-4cb9-8d91-d3a0f2f74881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:20.503036 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39453d28-166e-4cb9-8d91-d3a0f2f74881 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:20.504416 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39453d28-166e-4cb9-8d91-d3a0f2f74881 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:20.506551 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39453d28-166e-4cb9-8d91-d3a0f2f74881 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:20.536342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1052/2107] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:30:20 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:30:20.540498 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1061ca-accc-497f-bcb1-9fb0a7401efa None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:20.552274 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1061ca-accc-497f-bcb1-9fb0a7401efa None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:20.610140 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d35d3-4208-4ea7-85dc-db25878c1dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:20.610761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d35d3-4208-4ea7-85dc-db25878c1dfa admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:20.611042 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d35d3-4208-4ea7-85dc-db25878c1dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:20.611316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43d35d3-4208-4ea7-85dc-db25878c1dfa admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:20.611975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d35d3-4208-4ea7-85dc-db25878c1dfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:20.614440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43d35d3-4208-4ea7-85dc-db25878c1dfa admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:20.642068 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1056/2108] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:30:20 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:30:20.643280 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39453d28-166e-4cb9-8d91-d3a0f2f74881 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:20.654143 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39453d28-166e-4cb9-8d91-d3a0f2f74881 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:20.659750 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40397a86-c1c8-48bc-9b9e-c5d66af8a146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:20.660442 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40397a86-c1c8-48bc-9b9e-c5d66af8a146 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:20.660764 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40397a86-c1c8-48bc-9b9e-c5d66af8a146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:20.661081 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40397a86-c1c8-48bc-9b9e-c5d66af8a146 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:20.661977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40397a86-c1c8-48bc-9b9e-c5d66af8a146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:20.663758 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40397a86-c1c8-48bc-9b9e-c5d66af8a146 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:20.677559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1053/2109] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:30:20.680111 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d35d3-4208-4ea7-85dc-db25878c1dfa admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:20.691647 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43d35d3-4208-4ea7-85dc-db25878c1dfa admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:20.698031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b72c20-ae22-4d92-9122-cc8b2827c32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:20.698715 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b72c20-ae22-4d92-9122-cc8b2827c32a admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:20.699004 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b72c20-ae22-4d92-9122-cc8b2827c32a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:20.699285 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b72c20-ae22-4d92-9122-cc8b2827c32a admin admin] PATH_INFO: `/v3/projects/5ff2d240c3384e218ec062e3aeaccd48/users/8c0e0df4135143e384b4af2ae0ba7a4b/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:20.715580 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b72c20-ae22-4d92-9122-cc8b2827c32a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ff2d240c3384e218ec062e3aeaccd48, user_id=8c0e0df4135143e384b4af2ae0ba7a4b, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:20.719308 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b72c20-ae22-4d92-9122-cc8b2827c32a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:20.733826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1057/2110] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:30:20 2022] PUT /identity/v3/projects/5ff2d240c3384e218ec062e3aeaccd48/users/8c0e0df4135143e384b4af2ae0ba7a4b/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:30:20.737947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40397a86-c1c8-48bc-9b9e-c5d66af8a146 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:20.748730 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40397a86-c1c8-48bc-9b9e-c5d66af8a146 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:20.762043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:20.813511 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02868a2a-7ea0-4017-8689-3d882f0cb3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:20.814208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02868a2a-7ea0-4017-8689-3d882f0cb3b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:20.814537 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02868a2a-7ea0-4017-8689-3d882f0cb3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:20.814811 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02868a2a-7ea0-4017-8689-3d882f0cb3b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:20.815514 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02868a2a-7ea0-4017-8689-3d882f0cb3b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:20.818708 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02868a2a-7ea0-4017-8689-3d882f0cb3b4 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:20.829399 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1054/2111] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:30:20.833848 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b72c20-ae22-4d92-9122-cc8b2827c32a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:20.844080 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b72c20-ae22-4d92-9122-cc8b2827c32a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:20.848787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a6e2d-6895-4b77-8bda-e894837e1893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:20.849452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a6e2d-6895-4b77-8bda-e894837e1893 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:20.849782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a6e2d-6895-4b77-8bda-e894837e1893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:20.850073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a6e2d-6895-4b77-8bda-e894837e1893 admin admin] PATH_INFO: `/v3/projects/5ff2d240c3384e218ec062e3aeaccd48/users/8c0e0df4135143e384b4af2ae0ba7a4b/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:20.852098 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a6e2d-6895-4b77-8bda-e894837e1893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ff2d240c3384e218ec062e3aeaccd48, user_id=8c0e0df4135143e384b4af2ae0ba7a4b, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:20.854802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a6e2d-6895-4b77-8bda-e894837e1893 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:20.867628 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1058/2112] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:30:20 2022] PUT /identity/v3/projects/5ff2d240c3384e218ec062e3aeaccd48/users/8c0e0df4135143e384b4af2ae0ba7a4b/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:30:20.878066 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773edd0-ed89-41cc-a051-5e9693fd8ccb None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:20.878415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773edd0-ed89-41cc-a051-5e9693fd8ccb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:20.879340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1773edd0-ed89-41cc-a051-5e9693fd8ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:20.909300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1773edd0-ed89-41cc-a051-5e9693fd8ccb None None] MFA Rules not processed for user `8c0e0df4135143e384b4af2ae0ba7a4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:20.910580 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1773edd0-ed89-41cc-a051-5e9693fd8ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:20.987098 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1055/2113] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:20 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:30:20.998925 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a6e2d-6895-4b77-8bda-e894837e1893 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:21.000803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7d4a6e2d-6895-4b77-8bda-e894837e1893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:21.060044 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a6e2d-6895-4b77-8bda-e894837e1893 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:21.063763 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc24fe-6fcf-4974-9d8e-900f6f57e75e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:21.064378 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc24fe-6fcf-4974-9d8e-900f6f57e75e service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:21.064663 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc24fe-6fcf-4974-9d8e-900f6f57e75e service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:21.064938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc24fe-6fcf-4974-9d8e-900f6f57e75e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:21.074125 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc24fe-6fcf-4974-9d8e-900f6f57e75e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:21.075563 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfc24fe-6fcf-4974-9d8e-900f6f57e75e service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:21.085356 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1059/2114] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:30:20 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:30:21.205629 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773edd0-ed89-41cc-a051-5e9693fd8ccb None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:21.215271 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1773edd0-ed89-41cc-a051-5e9693fd8ccb None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:21.219115 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ecd7d9-bd8a-4958-88a1-3ad62a8bd8f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:21.219735 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecd7d9-bd8a-4958-88a1-3ad62a8bd8f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:21.220016 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecd7d9-bd8a-4958-88a1-3ad62a8bd8f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:21.220289 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ecd7d9-bd8a-4958-88a1-3ad62a8bd8f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:21.279751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ecd7d9-bd8a-4958-88a1-3ad62a8bd8f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:21.281104 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ecd7d9-bd8a-4958-88a1-3ad62a8bd8f7 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:21.290622 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1056/2115] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:30:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:30:31.080892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=76,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:31.286704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=83 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:37.780918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc24fe-6fcf-4974-9d8e-900f6f57e75e service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:37.881425 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc24fe-6fcf-4974-9d8e-900f6f57e75e service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:37.886662 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055ea36-9bd7-45cf-b26c-975fb83ae952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:37.887460 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055ea36-9bd7-45cf-b26c-975fb83ae952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:37.887662 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055ea36-9bd7-45cf-b26c-975fb83ae952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:37.888555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1055ea36-9bd7-45cf-b26c-975fb83ae952 admin admin] PATH_INFO: `/v3/users/756ba921e64d4a97a94b77b68940b0fe` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:37.889793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055ea36-9bd7-45cf-b26c-975fb83ae952 admin admin] RBAC: Authorizing `identity:delete_user(user_id=756ba921e64d4a97a94b77b68940b0fe)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:37.891729 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1055ea36-9bd7-45cf-b26c-975fb83ae952 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:37.955163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1055ea36-9bd7-45cf-b26c-975fb83ae952 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '756ba921e64d4a97a94b77b68940b0fe'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:37.966175 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1055ea36-9bd7-45cf-b26c-975fb83ae952 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '756ba921e64d4a97a94b77b68940b0fe'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:37.976438 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1055ea36-9bd7-45cf-b26c-975fb83ae952 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '756ba921e64d4a97a94b77b68940b0fe'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:37.978431 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1055ea36-9bd7-45cf-b26c-975fb83ae952 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '756ba921e64d4a97a94b77b68940b0fe'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:37.996476 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1055ea36-9bd7-45cf-b26c-975fb83ae952 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '756ba921e64d4a97a94b77b68940b0fe'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:38.013351 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1060/2116] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:30:37 2022] DELETE /identity/v3/users/756ba921e64d4a97a94b77b68940b0fe => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:30:38.015985 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad691f-2eed-47bb-9b73-b37e5e73fe57 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:38.016403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad691f-2eed-47bb-9b73-b37e5e73fe57 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:38.016499 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad691f-2eed-47bb-9b73-b37e5e73fe57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:38.045979 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d5ad691f-2eed-47bb-9b73-b37e5e73fe57 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:38.047632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d5ad691f-2eed-47bb-9b73-b37e5e73fe57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:38.127091 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1057/2117] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:30:38.142396 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055ea36-9bd7-45cf-b26c-975fb83ae952 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:38.143998 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1055ea36-9bd7-45cf-b26c-975fb83ae952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:38.227388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1055ea36-9bd7-45cf-b26c-975fb83ae952 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:38.231888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d4ab5-5894-4b44-bca2-2e2b72a6a385 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:38.232516 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d4ab5-5894-4b44-bca2-2e2b72a6a385 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:38.232797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d4ab5-5894-4b44-bca2-2e2b72a6a385 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:38.233073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d4ab5-5894-4b44-bca2-2e2b72a6a385 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:38.243393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d4ab5-5894-4b44-bca2-2e2b72a6a385 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:38.244923 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d4ab5-5894-4b44-bca2-2e2b72a6a385 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:38.256633 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1061/2118] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:30:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:30:38.664477 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad691f-2eed-47bb-9b73-b37e5e73fe57 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:38.666032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d5ad691f-2eed-47bb-9b73-b37e5e73fe57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:38.682749 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad691f-2eed-47bb-9b73-b37e5e73fe57 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:38.687778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a1781-bd92-41a5-b768-2be7b77fd3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:38.688680 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a1781-bd92-41a5-b768-2be7b77fd3c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:38.688959 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a1781-bd92-41a5-b768-2be7b77fd3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:38.689230 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92a1781-bd92-41a5-b768-2be7b77fd3c3 admin admin] PATH_INFO: `/v3/projects/7a284e8caf29431cbeddecbb71d03594` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:38.690593 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a1781-bd92-41a5-b768-2be7b77fd3c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a284e8caf29431cbeddecbb71d03594)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:38.693786 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92a1781-bd92-41a5-b768-2be7b77fd3c3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:38.751893 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e92a1781-bd92-41a5-b768-2be7b77fd3c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a284e8caf29431cbeddecbb71d03594', 'request_id': 'req-e92a1781-bd92-41a5-b768-2be7b77fd3c3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:38.759991 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e92a1781-bd92-41a5-b768-2be7b77fd3c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a284e8caf29431cbeddecbb71d03594', 'request_id': 'req-e92a1781-bd92-41a5-b768-2be7b77fd3c3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:38.766366 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e92a1781-bd92-41a5-b768-2be7b77fd3c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a284e8caf29431cbeddecbb71d03594', 'request_id': 'req-e92a1781-bd92-41a5-b768-2be7b77fd3c3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:38.781033 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e92a1781-bd92-41a5-b768-2be7b77fd3c3 admin admin] The token cache is being invalidate because project 7a284e8caf29431cbeddecbb71d03594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:30:38.781328 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e92a1781-bd92-41a5-b768-2be7b77fd3c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:38.782079 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e92a1781-bd92-41a5-b768-2be7b77fd3c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:38.783887 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1058/2119] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:30:38 2022] DELETE /identity/v3/projects/7a284e8caf29431cbeddecbb71d03594 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:30:38.788377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d4ab5-5894-4b44-bca2-2e2b72a6a385 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:38.799036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d4ab5-5894-4b44-bca2-2e2b72a6a385 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:38.823354 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:38.892815 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9352df72-dc1d-4ad0-93b4-57dc024f49c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:38.893500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352df72-dc1d-4ad0-93b4-57dc024f49c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:38.893827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352df72-dc1d-4ad0-93b4-57dc024f49c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:38.894878 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9352df72-dc1d-4ad0-93b4-57dc024f49c6 admin admin] PATH_INFO: `/v3/users/8c0e0df4135143e384b4af2ae0ba7a4b` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:38.896271 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352df72-dc1d-4ad0-93b4-57dc024f49c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c0e0df4135143e384b4af2ae0ba7a4b)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:38.898254 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9352df72-dc1d-4ad0-93b4-57dc024f49c6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:38.945393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9352df72-dc1d-4ad0-93b4-57dc024f49c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c0e0df4135143e384b4af2ae0ba7a4b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:38.952886 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9352df72-dc1d-4ad0-93b4-57dc024f49c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c0e0df4135143e384b4af2ae0ba7a4b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:38.960182 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9352df72-dc1d-4ad0-93b4-57dc024f49c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c0e0df4135143e384b4af2ae0ba7a4b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:38.960817 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9352df72-dc1d-4ad0-93b4-57dc024f49c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c0e0df4135143e384b4af2ae0ba7a4b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:38.973104 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9352df72-dc1d-4ad0-93b4-57dc024f49c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c0e0df4135143e384b4af2ae0ba7a4b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:38.979029 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1062/2120] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:30:38 2022] DELETE /identity/v3/users/8c0e0df4135143e384b4af2ae0ba7a4b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:30:39.276735 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a1781-bd92-41a5-b768-2be7b77fd3c3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:39.278462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e92a1781-bd92-41a5-b768-2be7b77fd3c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:39.294935 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92a1781-bd92-41a5-b768-2be7b77fd3c3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:39.380372 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175fada0-675a-40f2-8ff8-d182d62a7625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:39.380975 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fada0-675a-40f2-8ff8-d182d62a7625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:39.381361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fada0-675a-40f2-8ff8-d182d62a7625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:39.381682 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175fada0-675a-40f2-8ff8-d182d62a7625 admin admin] PATH_INFO: `/v3/projects/5ff2d240c3384e218ec062e3aeaccd48` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:39.383125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175fada0-675a-40f2-8ff8-d182d62a7625 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ff2d240c3384e218ec062e3aeaccd48)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:39.385084 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175fada0-675a-40f2-8ff8-d182d62a7625 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:39.430271 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:39.458841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-175fada0-675a-40f2-8ff8-d182d62a7625 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ff2d240c3384e218ec062e3aeaccd48', 'request_id': 'req-175fada0-675a-40f2-8ff8-d182d62a7625'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:39.467128 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-175fada0-675a-40f2-8ff8-d182d62a7625 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ff2d240c3384e218ec062e3aeaccd48', 'request_id': 'req-175fada0-675a-40f2-8ff8-d182d62a7625'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:39.477082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-175fada0-675a-40f2-8ff8-d182d62a7625 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ff2d240c3384e218ec062e3aeaccd48', 'request_id': 'req-175fada0-675a-40f2-8ff8-d182d62a7625'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:39.495629 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-175fada0-675a-40f2-8ff8-d182d62a7625 admin admin] The token cache is being invalidate because project 5ff2d240c3384e218ec062e3aeaccd48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:30:39.495935 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-175fada0-675a-40f2-8ff8-d182d62a7625 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:39.496563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-175fada0-675a-40f2-8ff8-d182d62a7625 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:30:39.498275 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1059/2121] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:30:39 2022] DELETE /identity/v3/projects/5ff2d240c3384e218ec062e3aeaccd48 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:30:39.503025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac071ac-cf22-430a-9f2a-31e780989f49 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:39.503393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac071ac-cf22-430a-9f2a-31e780989f49 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:39.503695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac071ac-cf22-430a-9f2a-31e780989f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:39.525759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-eac071ac-cf22-430a-9f2a-31e780989f49 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:39.527002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-eac071ac-cf22-430a-9f2a-31e780989f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:39.604697 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1063/2122] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:30:39.606341 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a4036-0931-48af-9942-3e88a2f5de4b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:39.606647 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a4036-0931-48af-9942-3e88a2f5de4b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:39.607162 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a4036-0931-48af-9942-3e88a2f5de4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:39.650049 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-257a4036-0931-48af-9942-3e88a2f5de4b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:39.651279 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-257a4036-0931-48af-9942-3e88a2f5de4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:39.662447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1060/2123] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:30:39.667641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac071ac-cf22-430a-9f2a-31e780989f49 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:39.697055 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac071ac-cf22-430a-9f2a-31e780989f49 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:39.705909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20ee6a-4cd8-4688-b24f-857fefdc3166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:39.706576 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20ee6a-4cd8-4688-b24f-857fefdc3166 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:39.706864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20ee6a-4cd8-4688-b24f-857fefdc3166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:39.707145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc20ee6a-4cd8-4688-b24f-857fefdc3166 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:39.707968 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20ee6a-4cd8-4688-b24f-857fefdc3166 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:30:39.708281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20ee6a-4cd8-4688-b24f-857fefdc3166 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:39.709921 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc20ee6a-4cd8-4688-b24f-857fefdc3166 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:39.745592 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a4036-0931-48af-9942-3e88a2f5de4b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:39.747277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1064/2124] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:30:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:30:39.760866 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a4036-0931-48af-9942-3e88a2f5de4b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:39.769361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e84dd9-4145-4a14-bf8f-9fad520016f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:39.770097 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e84dd9-4145-4a14-bf8f-9fad520016f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:39.770335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e84dd9-4145-4a14-bf8f-9fad520016f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:39.771201 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e84dd9-4145-4a14-bf8f-9fad520016f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:39.771986 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e84dd9-4145-4a14-bf8f-9fad520016f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:39.774837 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e84dd9-4145-4a14-bf8f-9fad520016f1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:39.830172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1061/2125] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:30:39 2022] POST /identity/v3/projects => generated 377 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:30:39.836498 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20ee6a-4cd8-4688-b24f-857fefdc3166 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:39.847824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc20ee6a-4cd8-4688-b24f-857fefdc3166 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:39.889366 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:39.946679 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f43e0-3bf7-40a8-bba5-1df5d184d7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:39.947379 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f43e0-3bf7-40a8-bba5-1df5d184d7d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:39.947665 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f43e0-3bf7-40a8-bba5-1df5d184d7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:39.947939 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f43e0-3bf7-40a8-bba5-1df5d184d7d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:39.948617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f43e0-3bf7-40a8-bba5-1df5d184d7d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:39.950612 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f43e0-3bf7-40a8-bba5-1df5d184d7d0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:39.985904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1065/2126] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:30:39 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:30:39.989129 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e84dd9-4145-4a14-bf8f-9fad520016f1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:39.999665 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e84dd9-4145-4a14-bf8f-9fad520016f1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:40.006278 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fcfeb2-ce37-42b1-88b8-075096780ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:40.007134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fcfeb2-ce37-42b1-88b8-075096780ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:40.007492 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fcfeb2-ce37-42b1-88b8-075096780ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:40.007839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fcfeb2-ce37-42b1-88b8-075096780ea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:40.008884 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fcfeb2-ce37-42b1-88b8-075096780ea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:40.010732 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fcfeb2-ce37-42b1-88b8-075096780ea9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:40.025443 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1062/2127] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:30:39 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:30:40.026197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f43e0-3bf7-40a8-bba5-1df5d184d7d0 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:40.040290 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f43e0-3bf7-40a8-bba5-1df5d184d7d0 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:40.045683 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611b473-bdba-4e11-b161-c5129d1edef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:40.046557 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611b473-bdba-4e11-b161-c5129d1edef6 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:40.046841 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611b473-bdba-4e11-b161-c5129d1edef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:40.047115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611b473-bdba-4e11-b161-c5129d1edef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:40.047826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611b473-bdba-4e11-b161-c5129d1edef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:40.049511 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611b473-bdba-4e11-b161-c5129d1edef6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:40.061870 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1066/2128] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:30:40.065663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fcfeb2-ce37-42b1-88b8-075096780ea9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:40.081392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fcfeb2-ce37-42b1-88b8-075096780ea9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:40.087718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b1e60-fb6c-45e5-867f-1952308b988f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:40.088645 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b1e60-fb6c-45e5-867f-1952308b988f admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:40.089117 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b1e60-fb6c-45e5-867f-1952308b988f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:40.089598 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753b1e60-fb6c-45e5-867f-1952308b988f admin admin] PATH_INFO: `/v3/projects/f4c04d6976e14d9ea66eccb0d94f5fb7/users/5d189023ae4e4045986af04731c28ce3/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:40.111149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b1e60-fb6c-45e5-867f-1952308b988f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4c04d6976e14d9ea66eccb0d94f5fb7, user_id=5d189023ae4e4045986af04731c28ce3, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:40.114411 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753b1e60-fb6c-45e5-867f-1952308b988f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:40.129806 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1063/2129] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:30:40 2022] PUT /identity/v3/projects/f4c04d6976e14d9ea66eccb0d94f5fb7/users/5d189023ae4e4045986af04731c28ce3/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:30:40.140679 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9f9af-a2a0-48ec-a13b-1913b3b21f75 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:40.141343 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9f9af-a2a0-48ec-a13b-1913b3b21f75 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:40.141848 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f9f9af-a2a0-48ec-a13b-1913b3b21f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:40.184507 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b7f9f9af-a2a0-48ec-a13b-1913b3b21f75 None None] MFA Rules not processed for user `5d189023ae4e4045986af04731c28ce3`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:40.185780 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b7f9f9af-a2a0-48ec-a13b-1913b3b21f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:40.266956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1067/2130] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:30:40.271772 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b1e60-fb6c-45e5-867f-1952308b988f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:40.344084 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753b1e60-fb6c-45e5-867f-1952308b988f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:40.349500 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8451b3-55e3-4dea-bdd1-8783665aefd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:40.350229 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8451b3-55e3-4dea-bdd1-8783665aefd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:40.350583 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8451b3-55e3-4dea-bdd1-8783665aefd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:40.350950 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8451b3-55e3-4dea-bdd1-8783665aefd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:40.351698 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8451b3-55e3-4dea-bdd1-8783665aefd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:40.353852 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8451b3-55e3-4dea-bdd1-8783665aefd3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:40.397892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1064/2131] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:30:40 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:30:40.402092 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9f9af-a2a0-48ec-a13b-1913b3b21f75 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:40.413819 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f9f9af-a2a0-48ec-a13b-1913b3b21f75 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:40.498956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d35cc-ad65-48ee-bda8-109da04a25fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:40.499561 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d35cc-ad65-48ee-bda8-109da04a25fe admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:40.499768 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d35cc-ad65-48ee-bda8-109da04a25fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:40.500380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18d35cc-ad65-48ee-bda8-109da04a25fe admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:40.500896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18d35cc-ad65-48ee-bda8-109da04a25fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:40.503090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18d35cc-ad65-48ee-bda8-109da04a25fe admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:40.532135 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1068/2132] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:30:40 2022] POST /identity/v3/users => generated 294 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:30:40.536839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8451b3-55e3-4dea-bdd1-8783665aefd3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:40.552930 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8451b3-55e3-4dea-bdd1-8783665aefd3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:40.558149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7a73e-4238-4476-bd19-483053eff0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:40.561047 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7a73e-4238-4476-bd19-483053eff0f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:40.561496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7a73e-4238-4476-bd19-483053eff0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:40.561802 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7a73e-4238-4476-bd19-483053eff0f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:40.562513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f7a73e-4238-4476-bd19-483053eff0f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:40.564132 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f7a73e-4238-4476-bd19-483053eff0f4 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:40.576540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1065/2133] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:30:40.583946 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d35cc-ad65-48ee-bda8-109da04a25fe admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:40.594651 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18d35cc-ad65-48ee-bda8-109da04a25fe admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:40.601156 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f029a25-9437-4a19-abd9-8b37f5f42a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:40.601333 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f029a25-9437-4a19-abd9-8b37f5f42a59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:40.601602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f029a25-9437-4a19-abd9-8b37f5f42a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:40.601892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f029a25-9437-4a19-abd9-8b37f5f42a59 admin admin] PATH_INFO: `/v3/projects/9a928dc136ef4052825a51c4d523827e/users/b4abd84253ef48d6be6ddc244ba2ff47/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:40.619267 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f029a25-9437-4a19-abd9-8b37f5f42a59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a928dc136ef4052825a51c4d523827e, user_id=b4abd84253ef48d6be6ddc244ba2ff47, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:40.621825 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f029a25-9437-4a19-abd9-8b37f5f42a59 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:40.640031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1069/2134] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:30:40 2022] PUT /identity/v3/projects/9a928dc136ef4052825a51c4d523827e/users/b4abd84253ef48d6be6ddc244ba2ff47/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:30:40.645266 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7a73e-4238-4476-bd19-483053eff0f4 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:40.658313 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f7a73e-4238-4476-bd19-483053eff0f4 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:40.681170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:40.728166 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4626438-3486-42ac-ac83-d28f20136542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:40.729015 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4626438-3486-42ac-ac83-d28f20136542 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:40.729142 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4626438-3486-42ac-ac83-d28f20136542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:40.729350 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4626438-3486-42ac-ac83-d28f20136542 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:40.730040 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4626438-3486-42ac-ac83-d28f20136542 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:40.731633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4626438-3486-42ac-ac83-d28f20136542 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:40.742607 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1066/2135] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:30:40.746788 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f029a25-9437-4a19-abd9-8b37f5f42a59 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:40.752729 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:40.762305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f029a25-9437-4a19-abd9-8b37f5f42a59 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:40.768300 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa5dc4-7da3-4d40-9331-4494377cbfe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:40.769668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa5dc4-7da3-4d40-9331-4494377cbfe0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:40.770225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa5dc4-7da3-4d40-9331-4494377cbfe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:40.770469 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa5dc4-7da3-4d40-9331-4494377cbfe0 admin admin] PATH_INFO: `/v3/projects/9a928dc136ef4052825a51c4d523827e/users/b4abd84253ef48d6be6ddc244ba2ff47/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:40.772893 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa5dc4-7da3-4d40-9331-4494377cbfe0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a928dc136ef4052825a51c4d523827e, user_id=b4abd84253ef48d6be6ddc244ba2ff47, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:40.775595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa5dc4-7da3-4d40-9331-4494377cbfe0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:40.791683 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1070/2136] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:30:40 2022] PUT /identity/v3/projects/9a928dc136ef4052825a51c4d523827e/users/b4abd84253ef48d6be6ddc244ba2ff47/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:30:40.804298 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08a84a-5200-48e8-81b2-948c44ad61ad None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:40.804692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08a84a-5200-48e8-81b2-948c44ad61ad None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:40.804980 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e08a84a-5200-48e8-81b2-948c44ad61ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:40.847362 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1e08a84a-5200-48e8-81b2-948c44ad61ad None None] MFA Rules not processed for user `b4abd84253ef48d6be6ddc244ba2ff47`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:40.849853 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1e08a84a-5200-48e8-81b2-948c44ad61ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:40.875859 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd76d7-90b3-4be3-897d-aefc26c73c56 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:40.876432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd76d7-90b3-4be3-897d-aefc26c73c56 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:40.876922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd76d7-90b3-4be3-897d-aefc26c73c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:40.929277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-76fd76d7-90b3-4be3-897d-aefc26c73c56 None None] MFA Rules not processed for user `85626350279c40ea99da93b9ed6994b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:40.932626 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-76fd76d7-90b3-4be3-897d-aefc26c73c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:40.961577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1067/2137] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:30:40.961577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff148b-1d65-4d36-a607-044b959eac88 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:40.961577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff148b-1d65-4d36-a607-044b959eac88 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:40.961577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff148b-1d65-4d36-a607-044b959eac88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:40.981219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-79ff148b-1d65-4d36-a607-044b959eac88 None None] MFA Rules not processed for user `afbab63bf079403b8ef652422d581d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:40.982704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-79ff148b-1d65-4d36-a607-044b959eac88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:41.011025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1071/2138] 10.222.0.39 () {58 vars in 1009 bytes} [Thu Jul 28 22:30:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:30:41.022893 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd76d7-90b3-4be3-897d-aefc26c73c56 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:41.024397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-76fd76d7-90b3-4be3-897d-aefc26c73c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:41.078501 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1068/2139] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:30:41.079370 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff148b-1d65-4d36-a607-044b959eac88 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:41.086635 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-79ff148b-1d65-4d36-a607-044b959eac88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:41.132306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd76d7-90b3-4be3-897d-aefc26c73c56 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:41.147221 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0c134-cf13-4e05-b399-40ebc4e2cbce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:41.147925 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0c134-cf13-4e05-b399-40ebc4e2cbce service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:41.148305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0c134-cf13-4e05-b399-40ebc4e2cbce service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:41.148607 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0c134-cf13-4e05-b399-40ebc4e2cbce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:41.179924 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:41.201216 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff148b-1d65-4d36-a607-044b959eac88 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:41.208872 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc869fad-09d2-4670-b3f7-13712947f1d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9107872f147f6ab8698b67bb545af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:41.209504 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc869fad-09d2-4670-b3f7-13712947f1d5 service swift] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:41.209816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc869fad-09d2-4670-b3f7-13712947f1d5 service swift] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:41.210090 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc869fad-09d2-4670-b3f7-13712947f1d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:41.219199 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc869fad-09d2-4670-b3f7-13712947f1d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:41.220804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc869fad-09d2-4670-b3f7-13712947f1d5 service swift] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:41.227622 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a0c134-cf13-4e05-b399-40ebc4e2cbce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:41.229066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a0c134-cf13-4e05-b399-40ebc4e2cbce service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:41.231245 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1069/2140] 10.222.0.39 () {60 vars in 1384 bytes} [Thu Jul 28 22:30:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 22:30:41.240672 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1072/2141] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:30:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:30:47.212663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc869fad-09d2-4670-b3f7-13712947f1d5 service swift] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:30:47.224934 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc869fad-09d2-4670-b3f7-13712947f1d5 service swift] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:30:47.284569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a977f-94be-4501-b7f9-f40b0993ef63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:30:47.284569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a977f-94be-4501-b7f9-f40b0993ef63 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:47.284569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a977f-94be-4501-b7f9-f40b0993ef63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:47.284569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a977f-94be-4501-b7f9-f40b0993ef63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:47.306560 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a977f-94be-4501-b7f9-f40b0993ef63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:30:47.308147 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a977f-94be-4501-b7f9-f40b0993ef63 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:30:47.321105 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1070/2142] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:30:47 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:30:50.911352 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f28c5-d350-4692-930e-2ef5a2ff3c82 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:30:50.911935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f28c5-d350-4692-930e-2ef5a2ff3c82 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:30:50.912095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f28c5-d350-4692-930e-2ef5a2ff3c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:30:50.940790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5f3f28c5-d350-4692-930e-2ef5a2ff3c82 None None] MFA Rules not processed for user `d38b517910894f7fa4351a2120b466d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:30:50.942633 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5f3f28c5-d350-4692-930e-2ef5a2ff3c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:30:51.014223 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:30:51.031293 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1073/2143] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:30:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:30:57.314297 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=15 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:31:01.024520 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:31:07.588202 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a977f-94be-4501-b7f9-f40b0993ef63 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:07.590363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-143a977f-94be-4501-b7f9-f40b0993ef63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:07.737365 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a977f-94be-4501-b7f9-f40b0993ef63 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:07.742533 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff668285-04b7-4829-b5a9-8f31d037d4f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe003cd7bfbe4b92b1d64d300c3e7d8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:07.743188 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff668285-04b7-4829-b5a9-8f31d037d4f6 service placement] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:07.743475 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff668285-04b7-4829-b5a9-8f31d037d4f6 service placement] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:07.743783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff668285-04b7-4829-b5a9-8f31d037d4f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:07.745617 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ff668285-04b7-4829-b5a9-8f31d037d4f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:07.759626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff668285-04b7-4829-b5a9-8f31d037d4f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:07.761241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff668285-04b7-4829-b5a9-8f31d037d4f6 service placement] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:07.771981 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1071/2144] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:31:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:31:12.338826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71ab13-16a1-4bd7-a698-1acd87e8340c None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:12.339359 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71ab13-16a1-4bd7-a698-1acd87e8340c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:12.339359 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c71ab13-16a1-4bd7-a698-1acd87e8340c None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:12.340952 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1074/2145] 10.222.0.39 () {54 vars in 921 bytes} [Thu Jul 28 22:31:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:31:12.347772 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94701ae-a5c2-499f-846a-0a6bd1d029b4 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:12.348278 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94701ae-a5c2-499f-846a-0a6bd1d029b4 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:12.348368 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94701ae-a5c2-499f-846a-0a6bd1d029b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:12.370393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e94701ae-a5c2-499f-846a-0a6bd1d029b4 None None] MFA Rules not processed for user `77efe8046ea0468cb0c7c58c24fc4e75`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:31:12.371627 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e94701ae-a5c2-499f-846a-0a6bd1d029b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:12.466444 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1072/2146] 10.222.0.39 () {58 vars in 1018 bytes} [Thu Jul 28 22:31:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:31:12.479797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71ab13-16a1-4bd7-a698-1acd87e8340c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:12.487735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7c71ab13-16a1-4bd7-a698-1acd87e8340c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:12.614520 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c71ab13-16a1-4bd7-a698-1acd87e8340c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:12.618783 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfdbe7-d79b-4453-a8bc-ae7387ca42fb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fecb440c0874ede9f3cf544cfed93ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:12.622007 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfdbe7-d79b-4453-a8bc-ae7387ca42fb service glance] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:12.622320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfdbe7-d79b-4453-a8bc-ae7387ca42fb service glance] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:12.622609 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfdbe7-d79b-4453-a8bc-ae7387ca42fb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:12.635645 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdfdbe7-d79b-4453-a8bc-ae7387ca42fb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:12.637168 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdfdbe7-d79b-4453-a8bc-ae7387ca42fb service glance] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:12.650230 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1075/2147] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:31:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:31:12.740765 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b264277-da2a-4725-bbdd-78f38544c70f None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:12.741848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b264277-da2a-4725-bbdd-78f38544c70f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:12.742158 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b264277-da2a-4725-bbdd-78f38544c70f None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:12.743616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1073/2148] 10.222.0.39 () {54 vars in 921 bytes} [Thu Jul 28 22:31:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:31:12.748677 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a027d2-3809-44bf-910f-e7ea90842081 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:12.748968 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a027d2-3809-44bf-910f-e7ea90842081 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:12.749242 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a027d2-3809-44bf-910f-e7ea90842081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:12.771982 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-72a027d2-3809-44bf-910f-e7ea90842081 None None] MFA Rules not processed for user `09b96431ac98447283960019cadde8a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:31:12.773739 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-72a027d2-3809-44bf-910f-e7ea90842081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:12.832244 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1076/2149] 10.222.0.39 () {58 vars in 1018 bytes} [Thu Jul 28 22:31:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:31:12.844463 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b264277-da2a-4725-bbdd-78f38544c70f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:12.846555 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5b264277-da2a-4725-bbdd-78f38544c70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:12.861468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b264277-da2a-4725-bbdd-78f38544c70f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:12.866080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3ca50-70bc-41f2-954b-fc3da95136df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:12.866332 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3ca50-70bc-41f2-954b-fc3da95136df service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:12.866714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3ca50-70bc-41f2-954b-fc3da95136df service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:12.867122 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3ca50-70bc-41f2-954b-fc3da95136df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:12.875618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3ca50-70bc-41f2-954b-fc3da95136df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:12.878012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3ca50-70bc-41f2-954b-fc3da95136df service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:12.895921 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1074/2150] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:31:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:31:12.989420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f657f5a-5647-4b0d-801d-eed1db04c09e None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:12.990460 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f657f5a-5647-4b0d-801d-eed1db04c09e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:12.990806 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f657f5a-5647-4b0d-801d-eed1db04c09e None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:12.992407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1077/2151] 10.222.0.39 () {54 vars in 921 bytes} [Thu Jul 28 22:31:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:31:12.999646 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f986f4d-fe80-481e-9328-9809a4406afd None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:12.999935 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f986f4d-fe80-481e-9328-9809a4406afd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:13.000209 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f986f4d-fe80-481e-9328-9809a4406afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:13.025021 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9f986f4d-fe80-481e-9328-9809a4406afd None None] MFA Rules not processed for user `77efe8046ea0468cb0c7c58c24fc4e75`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:31:13.026293 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9f986f4d-fe80-481e-9328-9809a4406afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:13.035720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1075/2152] 10.222.0.39 () {58 vars in 1018 bytes} [Thu Jul 28 22:31:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:31:13.046467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f657f5a-5647-4b0d-801d-eed1db04c09e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:13.049703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5f657f5a-5647-4b0d-801d-eed1db04c09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:13.061228 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f657f5a-5647-4b0d-801d-eed1db04c09e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:13.065329 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a0219-e1f9-460b-941f-0db862e4d031 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:13.065965 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a0219-e1f9-460b-941f-0db862e4d031 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:13.066246 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a0219-e1f9-460b-941f-0db862e4d031 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:13.066519 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a0219-e1f9-460b-941f-0db862e4d031 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:13.076356 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a0219-e1f9-460b-941f-0db862e4d031 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:13.077777 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a0219-e1f9-460b-941f-0db862e4d031 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:13.088839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1078/2153] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:31:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:31:16.129137 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8077e-169d-4e04-81d8-cfda9e5713cb None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:16.129137 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8077e-169d-4e04-81d8-cfda9e5713cb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:16.129137 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb8077e-169d-4e04-81d8-cfda9e5713cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:16.166589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9fb8077e-169d-4e04-81d8-cfda9e5713cb None None] MFA Rules not processed for user `9d5b147068c2491c9e4c95bbf5d0cbdd`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:31:16.167958 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9fb8077e-169d-4e04-81d8-cfda9e5713cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:16.269655 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1076/2154] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:31:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:31:17.380909 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a0219-e1f9-460b-941f-0db862e4d031 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:17.494679 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a0219-e1f9-460b-941f-0db862e4d031 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:17.501985 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1737e6-3e14-49f9-8aed-beeecb17ad09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:17.502863 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1737e6-3e14-49f9-8aed-beeecb17ad09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:17.503760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1737e6-3e14-49f9-8aed-beeecb17ad09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:17.504648 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1737e6-3e14-49f9-8aed-beeecb17ad09 admin admin] PATH_INFO: `/v3/users/5d189023ae4e4045986af04731c28ce3` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:17.506774 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1737e6-3e14-49f9-8aed-beeecb17ad09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d189023ae4e4045986af04731c28ce3)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:17.509130 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1737e6-3e14-49f9-8aed-beeecb17ad09 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:17.572068 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:31:17.620488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ca1737e6-3e14-49f9-8aed-beeecb17ad09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d189023ae4e4045986af04731c28ce3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:17.634068 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ca1737e6-3e14-49f9-8aed-beeecb17ad09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d189023ae4e4045986af04731c28ce3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:17.645609 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ca1737e6-3e14-49f9-8aed-beeecb17ad09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d189023ae4e4045986af04731c28ce3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:17.646723 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ca1737e6-3e14-49f9-8aed-beeecb17ad09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d189023ae4e4045986af04731c28ce3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:17.665486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ca1737e6-3e14-49f9-8aed-beeecb17ad09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d189023ae4e4045986af04731c28ce3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:17.683651 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1079/2155] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:31:17 2022] DELETE /identity/v3/users/5d189023ae4e4045986af04731c28ce3 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:31:17.685856 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bc5cb-bf99-4eee-8b0c-6cbaf2739d1e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:17.687147 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bc5cb-bf99-4eee-8b0c-6cbaf2739d1e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:17.687634 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bc5cb-bf99-4eee-8b0c-6cbaf2739d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:17.748922 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:31:17.760060 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a26bc5cb-bf99-4eee-8b0c-6cbaf2739d1e None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:31:17.761509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a26bc5cb-bf99-4eee-8b0c-6cbaf2739d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:17.837676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1077/2156] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:31:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:31:17.850512 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1737e6-3e14-49f9-8aed-beeecb17ad09 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:17.852104 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ca1737e6-3e14-49f9-8aed-beeecb17ad09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:17.939316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1737e6-3e14-49f9-8aed-beeecb17ad09 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:17.944882 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed08e51-2383-4c76-a0e8-d4f1d698a452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:17.945687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed08e51-2383-4c76-a0e8-d4f1d698a452 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:17.946187 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed08e51-2383-4c76-a0e8-d4f1d698a452 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:17.946563 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed08e51-2383-4c76-a0e8-d4f1d698a452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:17.964839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed08e51-2383-4c76-a0e8-d4f1d698a452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:17.964839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed08e51-2383-4c76-a0e8-d4f1d698a452 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:17.974055 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1080/2157] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:31:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:31:18.395076 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26bc5cb-bf99-4eee-8b0c-6cbaf2739d1e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:18.396563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a26bc5cb-bf99-4eee-8b0c-6cbaf2739d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:18.408735 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26bc5cb-bf99-4eee-8b0c-6cbaf2739d1e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:18.413732 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9b72d-a64e-494c-b893-0ef4141fdf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:18.414392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9b72d-a64e-494c-b893-0ef4141fdf5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:18.414673 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9b72d-a64e-494c-b893-0ef4141fdf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:18.414950 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac9b72d-a64e-494c-b893-0ef4141fdf5c admin admin] PATH_INFO: `/v3/projects/f4c04d6976e14d9ea66eccb0d94f5fb7` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:18.416210 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9b72d-a64e-494c-b893-0ef4141fdf5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4c04d6976e14d9ea66eccb0d94f5fb7)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:18.418287 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac9b72d-a64e-494c-b893-0ef4141fdf5c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:18.478673 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-aac9b72d-a64e-494c-b893-0ef4141fdf5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4c04d6976e14d9ea66eccb0d94f5fb7', 'request_id': 'req-aac9b72d-a64e-494c-b893-0ef4141fdf5c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:18.486634 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-aac9b72d-a64e-494c-b893-0ef4141fdf5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4c04d6976e14d9ea66eccb0d94f5fb7', 'request_id': 'req-aac9b72d-a64e-494c-b893-0ef4141fdf5c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:18.494143 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-aac9b72d-a64e-494c-b893-0ef4141fdf5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4c04d6976e14d9ea66eccb0d94f5fb7', 'request_id': 'req-aac9b72d-a64e-494c-b893-0ef4141fdf5c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:18.509509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-aac9b72d-a64e-494c-b893-0ef4141fdf5c admin admin] The token cache is being invalidate because project f4c04d6976e14d9ea66eccb0d94f5fb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:31:18.509923 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-aac9b72d-a64e-494c-b893-0ef4141fdf5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:18.510657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-aac9b72d-a64e-494c-b893-0ef4141fdf5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:18.512475 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1078/2158] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:31:18 2022] DELETE /identity/v3/projects/f4c04d6976e14d9ea66eccb0d94f5fb7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:31:18.516497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed08e51-2383-4c76-a0e8-d4f1d698a452 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:18.527635 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed08e51-2383-4c76-a0e8-d4f1d698a452 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:18.593159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6362e0-4cf7-46c8-8f76-860ee4e45413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:18.594176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6362e0-4cf7-46c8-8f76-860ee4e45413 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:18.594559 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6362e0-4cf7-46c8-8f76-860ee4e45413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:18.594937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6362e0-4cf7-46c8-8f76-860ee4e45413 admin admin] PATH_INFO: `/v3/users/b4abd84253ef48d6be6ddc244ba2ff47` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:18.596278 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6362e0-4cf7-46c8-8f76-860ee4e45413 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4abd84253ef48d6be6ddc244ba2ff47)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:18.599184 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6362e0-4cf7-46c8-8f76-860ee4e45413 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:18.656890 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:31:18.660901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5b6362e0-4cf7-46c8-8f76-860ee4e45413 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4abd84253ef48d6be6ddc244ba2ff47'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:18.683054 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5b6362e0-4cf7-46c8-8f76-860ee4e45413 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4abd84253ef48d6be6ddc244ba2ff47'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:18.692713 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5b6362e0-4cf7-46c8-8f76-860ee4e45413 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4abd84253ef48d6be6ddc244ba2ff47'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:18.694048 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5b6362e0-4cf7-46c8-8f76-860ee4e45413 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4abd84253ef48d6be6ddc244ba2ff47'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:18.735713 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5b6362e0-4cf7-46c8-8f76-860ee4e45413 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4abd84253ef48d6be6ddc244ba2ff47'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:18.763086 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1081/2159] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:31:18 2022] DELETE /identity/v3/users/b4abd84253ef48d6be6ddc244ba2ff47 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:31:19.755423 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9b72d-a64e-494c-b893-0ef4141fdf5c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:19.761189 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-aac9b72d-a64e-494c-b893-0ef4141fdf5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:19.780634 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac9b72d-a64e-494c-b893-0ef4141fdf5c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:19.845742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3b3b3d-9ed5-4cf0-9332-0dff5a9a9f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:19.846383 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b3b3d-9ed5-4cf0-9332-0dff5a9a9f13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:19.846659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b3b3d-9ed5-4cf0-9332-0dff5a9a9f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:19.846992 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3b3b3d-9ed5-4cf0-9332-0dff5a9a9f13 admin admin] PATH_INFO: `/v3/projects/9a928dc136ef4052825a51c4d523827e` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:19.848336 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b3b3d-9ed5-4cf0-9332-0dff5a9a9f13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a928dc136ef4052825a51c4d523827e)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:19.850481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3b3b3d-9ed5-4cf0-9332-0dff5a9a9f13 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:19.898923 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:31:19.917700 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7e3b3b3d-9ed5-4cf0-9332-0dff5a9a9f13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a928dc136ef4052825a51c4d523827e', 'request_id': 'req-7e3b3b3d-9ed5-4cf0-9332-0dff5a9a9f13'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:19.929260 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7e3b3b3d-9ed5-4cf0-9332-0dff5a9a9f13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a928dc136ef4052825a51c4d523827e', 'request_id': 'req-7e3b3b3d-9ed5-4cf0-9332-0dff5a9a9f13'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:19.936410 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7e3b3b3d-9ed5-4cf0-9332-0dff5a9a9f13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a928dc136ef4052825a51c4d523827e', 'request_id': 'req-7e3b3b3d-9ed5-4cf0-9332-0dff5a9a9f13'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:19.951369 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7e3b3b3d-9ed5-4cf0-9332-0dff5a9a9f13 admin admin] The token cache is being invalidate because project 9a928dc136ef4052825a51c4d523827e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:31:19.951660 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7e3b3b3d-9ed5-4cf0-9332-0dff5a9a9f13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:19.952373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7e3b3b3d-9ed5-4cf0-9332-0dff5a9a9f13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:19.954894 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1079/2160] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:31:19 2022] DELETE /identity/v3/projects/9a928dc136ef4052825a51c4d523827e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:31:19.959132 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f7048-9844-4024-8ee0-5c72ac9b77a8 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:19.959654 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f7048-9844-4024-8ee0-5c72ac9b77a8 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:19.959752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f7048-9844-4024-8ee0-5c72ac9b77a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:19.982393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-028f7048-9844-4024-8ee0-5c72ac9b77a8 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:31:19.983686 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-028f7048-9844-4024-8ee0-5c72ac9b77a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:20.056701 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1082/2161] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:31:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:31:20.064694 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfe577-e023-480d-a3ce-d8afd44b7333 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:20.065057 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfe577-e023-480d-a3ce-d8afd44b7333 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:20.065436 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cfe577-e023-480d-a3ce-d8afd44b7333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:20.086451 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c4cfe577-e023-480d-a3ce-d8afd44b7333 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:31:20.087751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c4cfe577-e023-480d-a3ce-d8afd44b7333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:20.096050 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1080/2162] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:31:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:31:20.100603 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f7048-9844-4024-8ee0-5c72ac9b77a8 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:20.111427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f7048-9844-4024-8ee0-5c72ac9b77a8 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:20.118465 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bebf6-23cc-4084-980e-35b13c3e3a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:20.119115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bebf6-23cc-4084-980e-35b13c3e3a1e admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:20.119443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bebf6-23cc-4084-980e-35b13c3e3a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:20.119729 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bebf6-23cc-4084-980e-35b13c3e3a1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:20.120445 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1bebf6-23cc-4084-980e-35b13c3e3a1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:31:20.120753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1bebf6-23cc-4084-980e-35b13c3e3a1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:20.122423 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1bebf6-23cc-4084-980e-35b13c3e3a1e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:20.144981 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1083/2163] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:31:20.146066 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cfe577-e023-480d-a3ce-d8afd44b7333 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:20.160618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cfe577-e023-480d-a3ce-d8afd44b7333 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:20.165942 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951959f-0651-4925-95f2-208beedeb871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:20.166566 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951959f-0651-4925-95f2-208beedeb871 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:20.166916 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951959f-0651-4925-95f2-208beedeb871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:20.167266 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951959f-0651-4925-95f2-208beedeb871 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:20.168031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951959f-0651-4925-95f2-208beedeb871 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:20.170095 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951959f-0651-4925-95f2-208beedeb871 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:20.201925 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1081/2164] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:31:20 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:31:20.203326 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bebf6-23cc-4084-980e-35b13c3e3a1e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:20.214808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bebf6-23cc-4084-980e-35b13c3e3a1e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:20.283887 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b364fe2-b3df-4847-8e1d-3611a47b5fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:20.284869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b364fe2-b3df-4847-8e1d-3611a47b5fca admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:20.285159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b364fe2-b3df-4847-8e1d-3611a47b5fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:20.285438 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b364fe2-b3df-4847-8e1d-3611a47b5fca admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:20.286763 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b364fe2-b3df-4847-8e1d-3611a47b5fca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:20.288128 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b364fe2-b3df-4847-8e1d-3611a47b5fca admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:20.311874 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1084/2165] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:31:20 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:31:20.316584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951959f-0651-4925-95f2-208beedeb871 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:20.327592 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951959f-0651-4925-95f2-208beedeb871 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:20.333106 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd598e-f45a-48b2-8829-8f5a3451f274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:20.333874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd598e-f45a-48b2-8829-8f5a3451f274 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:20.334325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd598e-f45a-48b2-8829-8f5a3451f274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:20.334672 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd598e-f45a-48b2-8829-8f5a3451f274 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:20.335437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd598e-f45a-48b2-8829-8f5a3451f274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:20.337216 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acd598e-f45a-48b2-8829-8f5a3451f274 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:20.349585 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1082/2166] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:31:20.352266 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b364fe2-b3df-4847-8e1d-3611a47b5fca admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:20.362519 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b364fe2-b3df-4847-8e1d-3611a47b5fca admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:20.367291 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6703fd45-df00-4802-aa14-0c8e5ee4a8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:20.367914 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703fd45-df00-4802-aa14-0c8e5ee4a8db admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:20.368203 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703fd45-df00-4802-aa14-0c8e5ee4a8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:20.368485 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6703fd45-df00-4802-aa14-0c8e5ee4a8db admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:20.369188 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703fd45-df00-4802-aa14-0c8e5ee4a8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:20.370888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6703fd45-df00-4802-aa14-0c8e5ee4a8db admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:20.386645 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1085/2167] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:31:20.387721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd598e-f45a-48b2-8829-8f5a3451f274 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:20.399495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd598e-f45a-48b2-8829-8f5a3451f274 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:20.406100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6dba0-70db-4f63-b4b0-eef86722187e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:20.406550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6dba0-70db-4f63-b4b0-eef86722187e admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:20.406832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6dba0-70db-4f63-b4b0-eef86722187e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:20.407161 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d6dba0-70db-4f63-b4b0-eef86722187e admin admin] PATH_INFO: `/v3/projects/2bf82168d9534db885e2b08e8d710e2c/users/41ca8fda7ad8407393f761da4f1585e5/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:20.431272 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d6dba0-70db-4f63-b4b0-eef86722187e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bf82168d9534db885e2b08e8d710e2c, user_id=41ca8fda7ad8407393f761da4f1585e5, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:20.434280 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d6dba0-70db-4f63-b4b0-eef86722187e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:20.457590 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1083/2168] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:31:20 2022] PUT /identity/v3/projects/2bf82168d9534db885e2b08e8d710e2c/users/41ca8fda7ad8407393f761da4f1585e5/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:31:20.468034 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67343d6-a595-4d39-82d2-f74f0287ed81 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:20.468420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67343d6-a595-4d39-82d2-f74f0287ed81 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:20.468805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67343d6-a595-4d39-82d2-f74f0287ed81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:20.514217 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-a67343d6-a595-4d39-82d2-f74f0287ed81 None None] MFA Rules not processed for user `41ca8fda7ad8407393f761da4f1585e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:31:20.516133 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a67343d6-a595-4d39-82d2-f74f0287ed81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:20.532954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:31:20.624765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1086/2169] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:31:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:31:20.630786 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6dba0-70db-4f63-b4b0-eef86722187e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:20.722597 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d6dba0-70db-4f63-b4b0-eef86722187e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:20.729321 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522652b7-31e0-4878-9107-3ccbbdf271de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:20.730629 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522652b7-31e0-4878-9107-3ccbbdf271de admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:20.731020 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522652b7-31e0-4878-9107-3ccbbdf271de admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:20.731501 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522652b7-31e0-4878-9107-3ccbbdf271de admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:20.732248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522652b7-31e0-4878-9107-3ccbbdf271de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:20.734285 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522652b7-31e0-4878-9107-3ccbbdf271de admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:20.775253 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67343d6-a595-4d39-82d2-f74f0287ed81 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:20.787182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1084/2170] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:31:20 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:31:20.791687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67343d6-a595-4d39-82d2-f74f0287ed81 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:20.873742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c428d9-a48d-4d86-b64e-48aead7a715c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:20.876460 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c428d9-a48d-4d86-b64e-48aead7a715c admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:20.876974 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c428d9-a48d-4d86-b64e-48aead7a715c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:20.877649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c428d9-a48d-4d86-b64e-48aead7a715c admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:20.879410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c428d9-a48d-4d86-b64e-48aead7a715c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:20.882478 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c428d9-a48d-4d86-b64e-48aead7a715c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:20.927130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522652b7-31e0-4878-9107-3ccbbdf271de admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:20.938868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1087/2171] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:31:20 2022] POST /identity/v3/users => generated 292 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:31:20.942938 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522652b7-31e0-4878-9107-3ccbbdf271de admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:20.950016 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb3985-09b7-4a2b-a615-78cea1fe6e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:20.950665 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb3985-09b7-4a2b-a615-78cea1fe6e5b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:20.950952 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb3985-09b7-4a2b-a615-78cea1fe6e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:20.951269 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb3985-09b7-4a2b-a615-78cea1fe6e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:20.952888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb3985-09b7-4a2b-a615-78cea1fe6e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:20.954918 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb3985-09b7-4a2b-a615-78cea1fe6e5b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:20.968139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1085/2172] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:31:20.973634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c428d9-a48d-4d86-b64e-48aead7a715c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:20.985819 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c428d9-a48d-4d86-b64e-48aead7a715c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:20.991267 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdabf23a-173a-47ca-8112-f2372f1ca0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:20.992015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabf23a-173a-47ca-8112-f2372f1ca0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:20.992378 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabf23a-173a-47ca-8112-f2372f1ca0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:20.992997 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdabf23a-173a-47ca-8112-f2372f1ca0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:20.993835 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdabf23a-173a-47ca-8112-f2372f1ca0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:20.995571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdabf23a-173a-47ca-8112-f2372f1ca0e6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:21.013929 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb3985-09b7-4a2b-a615-78cea1fe6e5b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:21.022152 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1088/2173] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:31:20 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:31:21.026695 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb3985-09b7-4a2b-a615-78cea1fe6e5b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:21.040160 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd55b3-0391-4d4d-833a-65de27302af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:21.041015 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd55b3-0391-4d4d-833a-65de27302af8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:21.041410 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd55b3-0391-4d4d-833a-65de27302af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:21.042032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bd55b3-0391-4d4d-833a-65de27302af8 admin admin] PATH_INFO: `/v3/projects/1e8411ff965b44edb93a67b17d9fae71/users/e1d307bd82ed422f8e5b09f416f470af/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:21.078146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd55b3-0391-4d4d-833a-65de27302af8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8411ff965b44edb93a67b17d9fae71, user_id=e1d307bd82ed422f8e5b09f416f470af, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:21.081046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bd55b3-0391-4d4d-833a-65de27302af8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:21.131535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1086/2174] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:31:21 2022] PUT /identity/v3/projects/1e8411ff965b44edb93a67b17d9fae71/users/e1d307bd82ed422f8e5b09f416f470af/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:31:21.133941 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca025100-cf09-481f-ac1f-4138c1b9554b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:21.134238 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca025100-cf09-481f-ac1f-4138c1b9554b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:21.134521 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca025100-cf09-481f-ac1f-4138c1b9554b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:21.175210 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ca025100-cf09-481f-ac1f-4138c1b9554b None None] MFA Rules not processed for user `e1d307bd82ed422f8e5b09f416f470af`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:31:21.176433 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ca025100-cf09-481f-ac1f-4138c1b9554b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:21.313578 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1089/2175] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:31:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:31:21.318121 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd55b3-0391-4d4d-833a-65de27302af8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:21.334655 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:31:21.417571 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bd55b3-0391-4d4d-833a-65de27302af8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:21.423834 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81392a-2595-4091-9a7b-c0c09fc44d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:21.425791 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81392a-2595-4091-9a7b-c0c09fc44d11 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:21.426338 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81392a-2595-4091-9a7b-c0c09fc44d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:21.426808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c81392a-2595-4091-9a7b-c0c09fc44d11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:21.427827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81392a-2595-4091-9a7b-c0c09fc44d11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:21.431390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c81392a-2595-4091-9a7b-c0c09fc44d11 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:21.485830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1087/2176] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:31:21 2022] POST /identity/v3/projects => generated 387 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:31:21.487217 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca025100-cf09-481f-ac1f-4138c1b9554b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:21.502358 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca025100-cf09-481f-ac1f-4138c1b9554b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:21.554187 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:31:21.606442 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb2baf-3188-48c3-b46e-d89f64c048c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:21.608275 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb2baf-3188-48c3-b46e-d89f64c048c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:21.608275 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb2baf-3188-48c3-b46e-d89f64c048c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:21.608275 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb2baf-3188-48c3-b46e-d89f64c048c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:21.608515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb2baf-3188-48c3-b46e-d89f64c048c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:21.611058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb2baf-3188-48c3-b46e-d89f64c048c4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:21.645292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1090/2177] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:31:21 2022] POST /identity/v3/users => generated 300 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:31:21.646486 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81392a-2595-4091-9a7b-c0c09fc44d11 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:21.656910 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c81392a-2595-4091-9a7b-c0c09fc44d11 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:21.662859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63799d51-7aea-40f6-8384-db1e1028675c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:21.663615 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63799d51-7aea-40f6-8384-db1e1028675c admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:21.663900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63799d51-7aea-40f6-8384-db1e1028675c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:21.664201 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63799d51-7aea-40f6-8384-db1e1028675c admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:21.664933 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63799d51-7aea-40f6-8384-db1e1028675c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:21.666607 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63799d51-7aea-40f6-8384-db1e1028675c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:21.678978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1088/2178] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:31:21.683066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb2baf-3188-48c3-b46e-d89f64c048c4 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:21.704450 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb2baf-3188-48c3-b46e-d89f64c048c4 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:21.713520 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34e49e-368f-47a0-bfb0-d9323ecbd12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:21.714761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34e49e-368f-47a0-bfb0-d9323ecbd12a admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:21.715048 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34e49e-368f-47a0-bfb0-d9323ecbd12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:21.715334 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34e49e-368f-47a0-bfb0-d9323ecbd12a admin admin] PATH_INFO: `/v3/projects/97b303722061427abb9306dec8bce1dd/users/9b640db8c9bb4a27b86a9fd18c50fe03/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:21.734415 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34e49e-368f-47a0-bfb0-d9323ecbd12a admin admin] RBAC: Authorizing `identity:create_grant(project_id=97b303722061427abb9306dec8bce1dd, user_id=9b640db8c9bb4a27b86a9fd18c50fe03, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:21.737233 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34e49e-368f-47a0-bfb0-d9323ecbd12a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:21.757401 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1091/2179] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:31:21 2022] PUT /identity/v3/projects/97b303722061427abb9306dec8bce1dd/users/9b640db8c9bb4a27b86a9fd18c50fe03/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:31:21.761201 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63799d51-7aea-40f6-8384-db1e1028675c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:21.775825 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63799d51-7aea-40f6-8384-db1e1028675c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:21.852693 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e7328-6104-42bf-9fc8-a7ee643c3f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:21.853353 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e7328-6104-42bf-9fc8-a7ee643c3f72 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:21.853835 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e7328-6104-42bf-9fc8-a7ee643c3f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:21.854182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e7328-6104-42bf-9fc8-a7ee643c3f72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:21.855051 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e7328-6104-42bf-9fc8-a7ee643c3f72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:21.856641 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e7328-6104-42bf-9fc8-a7ee643c3f72 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:21.869649 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1089/2180] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:31:21.872235 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34e49e-368f-47a0-bfb0-d9323ecbd12a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:21.887084 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34e49e-368f-47a0-bfb0-d9323ecbd12a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:21.892639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6859ce1-ba87-4d24-ac8e-a770f5b18e1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:21.893323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6859ce1-ba87-4d24-ac8e-a770f5b18e1a admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:21.894473 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6859ce1-ba87-4d24-ac8e-a770f5b18e1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:21.894915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6859ce1-ba87-4d24-ac8e-a770f5b18e1a admin admin] PATH_INFO: `/v3/projects/97b303722061427abb9306dec8bce1dd/users/9b640db8c9bb4a27b86a9fd18c50fe03/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:21.897014 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6859ce1-ba87-4d24-ac8e-a770f5b18e1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=97b303722061427abb9306dec8bce1dd, user_id=9b640db8c9bb4a27b86a9fd18c50fe03, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:21.900100 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6859ce1-ba87-4d24-ac8e-a770f5b18e1a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:21.917627 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1092/2181] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:31:21 2022] PUT /identity/v3/projects/97b303722061427abb9306dec8bce1dd/users/9b640db8c9bb4a27b86a9fd18c50fe03/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:31:21.927232 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd9b76-945b-4015-a3dd-e27cb39ecbfb None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:21.927781 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd9b76-945b-4015-a3dd-e27cb39ecbfb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:21.928282 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd9b76-945b-4015-a3dd-e27cb39ecbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:21.968345 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-fdfd9b76-945b-4015-a3dd-e27cb39ecbfb None None] MFA Rules not processed for user `9b640db8c9bb4a27b86a9fd18c50fe03`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:31:21.969827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fdfd9b76-945b-4015-a3dd-e27cb39ecbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:22.044668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1090/2182] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:31:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:31:22.056378 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6859ce1-ba87-4d24-ac8e-a770f5b18e1a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:22.058209 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d6859ce1-ba87-4d24-ac8e-a770f5b18e1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:22.135983 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6859ce1-ba87-4d24-ac8e-a770f5b18e1a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:22.139972 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464109ad-782e-4f73-aaf8-fabc2fa5e7e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:22.140790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464109ad-782e-4f73-aaf8-fabc2fa5e7e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:22.141285 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464109ad-782e-4f73-aaf8-fabc2fa5e7e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:22.141907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464109ad-782e-4f73-aaf8-fabc2fa5e7e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:22.154869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464109ad-782e-4f73-aaf8-fabc2fa5e7e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:22.156801 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464109ad-782e-4f73-aaf8-fabc2fa5e7e4 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:22.168790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1093/2183] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:31:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:31:22.818803 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd9b76-945b-4015-a3dd-e27cb39ecbfb None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:22.820339 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fdfd9b76-945b-4015-a3dd-e27cb39ecbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:22.828884 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:31:22.835173 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd9b76-945b-4015-a3dd-e27cb39ecbfb None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:22.839606 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018fc49-4eb0-4f5a-8236-7da9f8105bdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:22.840503 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018fc49-4eb0-4f5a-8236-7da9f8105bdd service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:22.841001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018fc49-4eb0-4f5a-8236-7da9f8105bdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:22.841621 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018fc49-4eb0-4f5a-8236-7da9f8105bdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:22.907206 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018fc49-4eb0-4f5a-8236-7da9f8105bdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:22.908671 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018fc49-4eb0-4f5a-8236-7da9f8105bdd service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:22.922263 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1091/2184] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:31:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:31:23.674816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2570a6b-558d-4cba-b7f9-565a043c6523 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:23.675317 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2570a6b-558d-4cba-b7f9-565a043c6523 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:23.675435 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2570a6b-558d-4cba-b7f9-565a043c6523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:23.695298 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f2570a6b-558d-4cba-b7f9-565a043c6523 None None] MFA Rules not processed for user `85626350279c40ea99da93b9ed6994b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:31:23.696528 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f2570a6b-558d-4cba-b7f9-565a043c6523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:23.758955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1094/2185] 10.222.0.39 () {58 vars in 1009 bytes} [Thu Jul 28 22:31:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:31:23.774137 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018fc49-4eb0-4f5a-8236-7da9f8105bdd service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:23.775767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9018fc49-4eb0-4f5a-8236-7da9f8105bdd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:23.850130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018fc49-4eb0-4f5a-8236-7da9f8105bdd service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:23.855362 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52df94c-1df6-4720-9344-3b23ab4f507b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9107872f147f6ab8698b67bb545af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:23.855983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52df94c-1df6-4720-9344-3b23ab4f507b service swift] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:23.856270 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52df94c-1df6-4720-9344-3b23ab4f507b service swift] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:23.856548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52df94c-1df6-4720-9344-3b23ab4f507b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:23.864946 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52df94c-1df6-4720-9344-3b23ab4f507b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:23.866617 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52df94c-1df6-4720-9344-3b23ab4f507b service swift] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:23.875046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1092/2186] 10.222.0.39 () {60 vars in 1384 bytes} [Thu Jul 28 22:31:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 22:31:24.860459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2570a6b-558d-4cba-b7f9-565a043c6523 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:24.863535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f2570a6b-558d-4cba-b7f9-565a043c6523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:24.903867 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:31:24.939513 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2570a6b-558d-4cba-b7f9-565a043c6523 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:24.944064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d0db2-1de5-4532-8719-9c23025e2641 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:24.945083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d0db2-1de5-4532-8719-9c23025e2641 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:24.945429 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d0db2-1de5-4532-8719-9c23025e2641 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:24.945826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7d0db2-1de5-4532-8719-9c23025e2641 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:24.947933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fd7d0db2-1de5-4532-8719-9c23025e2641 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:25.015284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d0db2-1de5-4532-8719-9c23025e2641 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:25.016752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7d0db2-1de5-4532-8719-9c23025e2641 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:25.026655 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1095/2187] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:31:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:31:27.452547 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52df94c-1df6-4720-9344-3b23ab4f507b service swift] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:27.470030 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52df94c-1df6-4720-9344-3b23ab4f507b service swift] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:27.475920 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b28540-d7f5-4e35-9b91-59fafac86f5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:27.476533 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b28540-d7f5-4e35-9b91-59fafac86f5d service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:27.476812 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b28540-d7f5-4e35-9b91-59fafac86f5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:27.477084 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b28540-d7f5-4e35-9b91-59fafac86f5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:27.577088 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b28540-d7f5-4e35-9b91-59fafac86f5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:27.578890 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b28540-d7f5-4e35-9b91-59fafac86f5d service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:27.596198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1093/2188] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:31:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:31:30.329773 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d0db2-1de5-4532-8719-9c23025e2641 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:30.398877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7d0db2-1de5-4532-8719-9c23025e2641 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:30.404971 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568c0f8-260a-469d-8546-30f56d45d43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:30.405879 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568c0f8-260a-469d-8546-30f56d45d43b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:30.406347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568c0f8-260a-469d-8546-30f56d45d43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:30.406812 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568c0f8-260a-469d-8546-30f56d45d43b admin admin] PATH_INFO: `/v3/users/41ca8fda7ad8407393f761da4f1585e5` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:30.408299 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568c0f8-260a-469d-8546-30f56d45d43b admin admin] RBAC: Authorizing `identity:delete_user(user_id=41ca8fda7ad8407393f761da4f1585e5)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:30.410927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568c0f8-260a-469d-8546-30f56d45d43b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:30.461319 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7568c0f8-260a-469d-8546-30f56d45d43b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41ca8fda7ad8407393f761da4f1585e5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:30.467949 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7568c0f8-260a-469d-8546-30f56d45d43b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41ca8fda7ad8407393f761da4f1585e5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:30.476032 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7568c0f8-260a-469d-8546-30f56d45d43b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41ca8fda7ad8407393f761da4f1585e5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:30.476927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7568c0f8-260a-469d-8546-30f56d45d43b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41ca8fda7ad8407393f761da4f1585e5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:30.489726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:31:30.492020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7568c0f8-260a-469d-8546-30f56d45d43b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41ca8fda7ad8407393f761da4f1585e5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:30.499641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1096/2189] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:31:30 2022] DELETE /identity/v3/users/41ca8fda7ad8407393f761da4f1585e5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:31:30.505277 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c4093-e82e-464d-b954-ea938970fc41 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:30.505820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c4093-e82e-464d-b954-ea938970fc41 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:30.505917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c4093-e82e-464d-b954-ea938970fc41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:30.528087 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-065c4093-e82e-464d-b954-ea938970fc41 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:31:30.530010 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-065c4093-e82e-464d-b954-ea938970fc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:30.581395 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:31:30.619679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1094/2190] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:31:30.629860 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568c0f8-260a-469d-8546-30f56d45d43b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:30.631526 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7568c0f8-260a-469d-8546-30f56d45d43b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:30.688438 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568c0f8-260a-469d-8546-30f56d45d43b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:30.692224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719841b-f43e-4b0b-9afb-d6a2dcbdb676 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:30.692839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719841b-f43e-4b0b-9afb-d6a2dcbdb676 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:30.693120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719841b-f43e-4b0b-9afb-d6a2dcbdb676 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:30.693470 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6719841b-f43e-4b0b-9afb-d6a2dcbdb676 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:30.706370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6719841b-f43e-4b0b-9afb-d6a2dcbdb676 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:30.708256 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6719841b-f43e-4b0b-9afb-d6a2dcbdb676 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:30.725730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1097/2191] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:31:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:31:31.114297 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c4093-e82e-464d-b954-ea938970fc41 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:31.115787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-065c4093-e82e-464d-b954-ea938970fc41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:31.128470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c4093-e82e-464d-b954-ea938970fc41 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:31.141406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ede6d0-c82b-4adf-b133-93b1e2efea6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:31.141897 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ede6d0-c82b-4adf-b133-93b1e2efea6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:31.142183 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ede6d0-c82b-4adf-b133-93b1e2efea6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:31.142458 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ede6d0-c82b-4adf-b133-93b1e2efea6a admin admin] PATH_INFO: `/v3/projects/2bf82168d9534db885e2b08e8d710e2c` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:31.144327 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ede6d0-c82b-4adf-b133-93b1e2efea6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bf82168d9534db885e2b08e8d710e2c)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:31.147112 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ede6d0-c82b-4adf-b133-93b1e2efea6a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:31.200889 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-62ede6d0-c82b-4adf-b133-93b1e2efea6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bf82168d9534db885e2b08e8d710e2c', 'request_id': 'req-62ede6d0-c82b-4adf-b133-93b1e2efea6a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:31.208807 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-62ede6d0-c82b-4adf-b133-93b1e2efea6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bf82168d9534db885e2b08e8d710e2c', 'request_id': 'req-62ede6d0-c82b-4adf-b133-93b1e2efea6a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:31.216043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-62ede6d0-c82b-4adf-b133-93b1e2efea6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bf82168d9534db885e2b08e8d710e2c', 'request_id': 'req-62ede6d0-c82b-4adf-b133-93b1e2efea6a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:31.230697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-62ede6d0-c82b-4adf-b133-93b1e2efea6a admin admin] The token cache is being invalidate because project 2bf82168d9534db885e2b08e8d710e2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:31:31.231060 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-62ede6d0-c82b-4adf-b133-93b1e2efea6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:31.231625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-62ede6d0-c82b-4adf-b133-93b1e2efea6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:31.233634 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1095/2192] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:31:31 2022] DELETE /identity/v3/projects/2bf82168d9534db885e2b08e8d710e2c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:31:31.238057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719841b-f43e-4b0b-9afb-d6a2dcbdb676 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:31.262116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6719841b-f43e-4b0b-9afb-d6a2dcbdb676 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:31.345865 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fe6cd-30db-4958-8694-a6a847a22d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:31.347278 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fe6cd-30db-4958-8694-a6a847a22d2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:31.347589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fe6cd-30db-4958-8694-a6a847a22d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:31.347868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fe6cd-30db-4958-8694-a6a847a22d2e admin admin] PATH_INFO: `/v3/users/e1d307bd82ed422f8e5b09f416f470af` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:31.354401 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fe6cd-30db-4958-8694-a6a847a22d2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1d307bd82ed422f8e5b09f416f470af)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:31.356406 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fe6cd-30db-4958-8694-a6a847a22d2e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:31.418032 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-333fe6cd-30db-4958-8694-a6a847a22d2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1d307bd82ed422f8e5b09f416f470af'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:31.428742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-333fe6cd-30db-4958-8694-a6a847a22d2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1d307bd82ed422f8e5b09f416f470af'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:31.436573 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-333fe6cd-30db-4958-8694-a6a847a22d2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1d307bd82ed422f8e5b09f416f470af'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:31.437315 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-333fe6cd-30db-4958-8694-a6a847a22d2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1d307bd82ed422f8e5b09f416f470af'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:31.449747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-333fe6cd-30db-4958-8694-a6a847a22d2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1d307bd82ed422f8e5b09f416f470af'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:31.455575 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1098/2193] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:31:31 2022] DELETE /identity/v3/users/e1d307bd82ed422f8e5b09f416f470af => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:31:31.804895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ede6d0-c82b-4adf-b133-93b1e2efea6a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:31.806511 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-62ede6d0-c82b-4adf-b133-93b1e2efea6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:31.820215 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ede6d0-c82b-4adf-b133-93b1e2efea6a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:31.880064 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c21b4-a7c5-4f62-a3f4-2d64715dc69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:31.880714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c21b4-a7c5-4f62-a3f4-2d64715dc69d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:31.881251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c21b4-a7c5-4f62-a3f4-2d64715dc69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:31.881541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c21b4-a7c5-4f62-a3f4-2d64715dc69d admin admin] PATH_INFO: `/v3/projects/1e8411ff965b44edb93a67b17d9fae71` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:31.882770 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c21b4-a7c5-4f62-a3f4-2d64715dc69d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e8411ff965b44edb93a67b17d9fae71)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:31.884779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c21b4-a7c5-4f62-a3f4-2d64715dc69d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:31.938092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-555c21b4-a7c5-4f62-a3f4-2d64715dc69d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e8411ff965b44edb93a67b17d9fae71', 'request_id': 'req-555c21b4-a7c5-4f62-a3f4-2d64715dc69d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:31.945245 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-555c21b4-a7c5-4f62-a3f4-2d64715dc69d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e8411ff965b44edb93a67b17d9fae71', 'request_id': 'req-555c21b4-a7c5-4f62-a3f4-2d64715dc69d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:31.946576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:31:31.955006 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-555c21b4-a7c5-4f62-a3f4-2d64715dc69d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e8411ff965b44edb93a67b17d9fae71', 'request_id': 'req-555c21b4-a7c5-4f62-a3f4-2d64715dc69d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:31.970577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-555c21b4-a7c5-4f62-a3f4-2d64715dc69d admin admin] The token cache is being invalidate because project 1e8411ff965b44edb93a67b17d9fae71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:31:31.970793 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-555c21b4-a7c5-4f62-a3f4-2d64715dc69d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:31.972091 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-555c21b4-a7c5-4f62-a3f4-2d64715dc69d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:31.973640 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1096/2194] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:31:31 2022] DELETE /identity/v3/projects/1e8411ff965b44edb93a67b17d9fae71 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:31:31.977754 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fe6cd-30db-4958-8694-a6a847a22d2e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:31.979175 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-333fe6cd-30db-4958-8694-a6a847a22d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:31.992437 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fe6cd-30db-4958-8694-a6a847a22d2e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:32.004212 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:31:32.086244 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872de5dd-8db5-4c21-80c9-a9251bb34695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:32.090120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872de5dd-8db5-4c21-80c9-a9251bb34695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:32.090745 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872de5dd-8db5-4c21-80c9-a9251bb34695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:32.091305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872de5dd-8db5-4c21-80c9-a9251bb34695 admin admin] PATH_INFO: `/v3/users/9b640db8c9bb4a27b86a9fd18c50fe03` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:32.093070 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872de5dd-8db5-4c21-80c9-a9251bb34695 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b640db8c9bb4a27b86a9fd18c50fe03)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:32.095502 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872de5dd-8db5-4c21-80c9-a9251bb34695 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:32.149238 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-872de5dd-8db5-4c21-80c9-a9251bb34695 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b640db8c9bb4a27b86a9fd18c50fe03'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:32.157685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-872de5dd-8db5-4c21-80c9-a9251bb34695 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b640db8c9bb4a27b86a9fd18c50fe03'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:32.165957 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-872de5dd-8db5-4c21-80c9-a9251bb34695 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b640db8c9bb4a27b86a9fd18c50fe03'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:32.166602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-872de5dd-8db5-4c21-80c9-a9251bb34695 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b640db8c9bb4a27b86a9fd18c50fe03'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:32.185677 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-872de5dd-8db5-4c21-80c9-a9251bb34695 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b640db8c9bb4a27b86a9fd18c50fe03'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:32.193403 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1099/2195] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:31:31 2022] DELETE /identity/v3/users/9b640db8c9bb4a27b86a9fd18c50fe03 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:31:32.671858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c21b4-a7c5-4f62-a3f4-2d64715dc69d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:32.675675 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-555c21b4-a7c5-4f62-a3f4-2d64715dc69d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:32.688226 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c21b4-a7c5-4f62-a3f4-2d64715dc69d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:32.756358 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950056fd-38ac-4d9c-9f1e-7848d497af57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:32.757004 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950056fd-38ac-4d9c-9f1e-7848d497af57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:32.757280 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950056fd-38ac-4d9c-9f1e-7848d497af57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:32.757575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950056fd-38ac-4d9c-9f1e-7848d497af57 admin admin] PATH_INFO: `/v3/projects/97b303722061427abb9306dec8bce1dd` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:32.758742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950056fd-38ac-4d9c-9f1e-7848d497af57 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97b303722061427abb9306dec8bce1dd)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:32.760697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950056fd-38ac-4d9c-9f1e-7848d497af57 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:32.841797 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-950056fd-38ac-4d9c-9f1e-7848d497af57 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97b303722061427abb9306dec8bce1dd', 'request_id': 'req-950056fd-38ac-4d9c-9f1e-7848d497af57'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:32.850082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-950056fd-38ac-4d9c-9f1e-7848d497af57 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97b303722061427abb9306dec8bce1dd', 'request_id': 'req-950056fd-38ac-4d9c-9f1e-7848d497af57'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:32.856070 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-950056fd-38ac-4d9c-9f1e-7848d497af57 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97b303722061427abb9306dec8bce1dd', 'request_id': 'req-950056fd-38ac-4d9c-9f1e-7848d497af57'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:32.869792 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-950056fd-38ac-4d9c-9f1e-7848d497af57 admin admin] The token cache is being invalidate because project 97b303722061427abb9306dec8bce1dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:31:32.870325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-950056fd-38ac-4d9c-9f1e-7848d497af57 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:32.871087 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-950056fd-38ac-4d9c-9f1e-7848d497af57 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:31:32.873356 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1097/2196] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:31:32 2022] DELETE /identity/v3/projects/97b303722061427abb9306dec8bce1dd => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:31:42.189934 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=46,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:31:42.866663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=10,keystone:SELECT=55,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:31:51.207370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6abc8-4c7e-4e5f-a137-68167a7d4e66 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:51.208615 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6abc8-4c7e-4e5f-a137-68167a7d4e66 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:51.208718 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6abc8-4c7e-4e5f-a137-68167a7d4e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:51.227887 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-50e6abc8-4c7e-4e5f-a137-68167a7d4e66 None None] MFA Rules not processed for user `85626350279c40ea99da93b9ed6994b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:31:51.229107 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-50e6abc8-4c7e-4e5f-a137-68167a7d4e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:51.283593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1100/2197] 10.222.0.39 () {58 vars in 967 bytes} [Thu Jul 28 22:31:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:31:51.306587 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950056fd-38ac-4d9c-9f1e-7848d497af57 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:31:51.308035 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-950056fd-38ac-4d9c-9f1e-7848d497af57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:31:51.377045 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950056fd-38ac-4d9c-9f1e-7848d497af57 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:31:51.382996 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349615f-a66f-4996-af0c-9a738766bfa5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9107872f147f6ab8698b67bb545af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:31:51.383604 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349615f-a66f-4996-af0c-9a738766bfa5 service swift] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:31:51.383889 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349615f-a66f-4996-af0c-9a738766bfa5 service swift] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:31:51.384167 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349615f-a66f-4996-af0c-9a738766bfa5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:31:51.392463 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349615f-a66f-4996-af0c-9a738766bfa5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:31:51.394075 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349615f-a66f-4996-af0c-9a738766bfa5 service swift] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:31:51.402877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1098/2198] 10.222.0.39 () {60 vars in 1384 bytes} [Thu Jul 28 22:31:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 22:32:01.276996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=25 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:32:01.399326 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=32 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:32:35.614485 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4e762-bd81-434c-96ec-cfbf5de6ab8d None None] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:35.615524 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4e762-bd81-434c-96ec-cfbf5de6ab8d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:35.615524 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee4e762-bd81-434c-96ec-cfbf5de6ab8d None None] PATH_INFO: `/` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:35.616931 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1101/2199] 10.222.0.39 () {54 vars in 921 bytes} [Thu Jul 28 22:32:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 28 22:32:35.644375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a90595-b800-4f45-95b5-a947238487d9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:35.644911 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a90595-b800-4f45-95b5-a947238487d9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:35.645019 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a90595-b800-4f45-95b5-a947238487d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:35.664922 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-67a90595-b800-4f45-95b5-a947238487d9 None None] MFA Rules not processed for user `fe003cd7bfbe4b92b1d64d300c3e7d8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:32:35.666341 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-67a90595-b800-4f45-95b5-a947238487d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:32:35.683254 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4e762-bd81-434c-96ec-cfbf5de6ab8d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:32:35.685036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-7ee4e762-bd81-434c-96ec-cfbf5de6ab8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:32:35.746352 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1099/2200] 10.222.0.39 () {58 vars in 1018 bytes} [Thu Jul 28 22:32:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:32:35.760577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a90595-b800-4f45-95b5-a947238487d9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:32:35.762185 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-67a90595-b800-4f45-95b5-a947238487d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:32:35.762660 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee4e762-bd81-434c-96ec-cfbf5de6ab8d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:32:35.767395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b750f-60cd-47d1-8242-6edfc473d239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:32:35.768036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b750f-60cd-47d1-8242-6edfc473d239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:35.768321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b750f-60cd-47d1-8242-6edfc473d239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:35.768597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b750f-60cd-47d1-8242-6edfc473d239 admin admin] PATH_INFO: `/v3/users/9d5b147068c2491c9e4c95bbf5d0cbdd` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:35.769819 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b750f-60cd-47d1-8242-6edfc473d239 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d5b147068c2491c9e4c95bbf5d0cbdd)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:32:35.772537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23b750f-60cd-47d1-8242-6edfc473d239 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:32:35.776310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a90595-b800-4f45-95b5-a947238487d9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:32:35.784687 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3421063-cfeb-406d-8a43-e75519754329 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe003cd7bfbe4b92b1d64d300c3e7d8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:32:35.785314 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3421063-cfeb-406d-8a43-e75519754329 service placement] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:35.785634 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3421063-cfeb-406d-8a43-e75519754329 service placement] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:35.785904 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3421063-cfeb-406d-8a43-e75519754329 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:35.795706 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3421063-cfeb-406d-8a43-e75519754329 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:32:35.797308 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3421063-cfeb-406d-8a43-e75519754329 service placement] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:32:35.808536 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1100/2201] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:32:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:32:35.838226 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d23b750f-60cd-47d1-8242-6edfc473d239 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d5b147068c2491c9e4c95bbf5d0cbdd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:32:35.848135 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d23b750f-60cd-47d1-8242-6edfc473d239 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d5b147068c2491c9e4c95bbf5d0cbdd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:32:35.856520 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d23b750f-60cd-47d1-8242-6edfc473d239 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d5b147068c2491c9e4c95bbf5d0cbdd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:32:35.857336 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d23b750f-60cd-47d1-8242-6edfc473d239 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d5b147068c2491c9e4c95bbf5d0cbdd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:32:35.876828 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d23b750f-60cd-47d1-8242-6edfc473d239 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d5b147068c2491c9e4c95bbf5d0cbdd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:32:35.884434 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1102/2202] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:32:35 2022] DELETE /identity/v3/users/9d5b147068c2491c9e4c95bbf5d0cbdd => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:32:36.071418 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3421063-cfeb-406d-8a43-e75519754329 service placement] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:32:36.083647 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3421063-cfeb-406d-8a43-e75519754329 service placement] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:32:36.143137 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e840df-0166-4b16-890f-da667624cd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:32:36.143706 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e840df-0166-4b16-890f-da667624cd3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:36.143990 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e840df-0166-4b16-890f-da667624cd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:36.144269 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e840df-0166-4b16-890f-da667624cd3e admin admin] PATH_INFO: `/v3/projects/da5c439035bf4fe1bcf123752ab0566f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:36.145426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e840df-0166-4b16-890f-da667624cd3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=da5c439035bf4fe1bcf123752ab0566f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:32:36.149826 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e840df-0166-4b16-890f-da667624cd3e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:32:36.199471 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-69e840df-0166-4b16-890f-da667624cd3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da5c439035bf4fe1bcf123752ab0566f', 'request_id': 'req-69e840df-0166-4b16-890f-da667624cd3e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:32:36.205405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-69e840df-0166-4b16-890f-da667624cd3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da5c439035bf4fe1bcf123752ab0566f', 'request_id': 'req-69e840df-0166-4b16-890f-da667624cd3e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:32:36.210865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-69e840df-0166-4b16-890f-da667624cd3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da5c439035bf4fe1bcf123752ab0566f', 'request_id': 'req-69e840df-0166-4b16-890f-da667624cd3e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:32:36.223171 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-69e840df-0166-4b16-890f-da667624cd3e admin admin] The token cache is being invalidate because project da5c439035bf4fe1bcf123752ab0566f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:32:36.223457 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-69e840df-0166-4b16-890f-da667624cd3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:32:36.224070 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-69e840df-0166-4b16-890f-da667624cd3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:32:36.225667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1101/2203] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:32:36 2022] DELETE /identity/v3/projects/da5c439035bf4fe1bcf123752ab0566f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:32:36.230878 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937bfea-1485-46b5-a877-d5ab3c5c06cd None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:36.231171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937bfea-1485-46b5-a877-d5ab3c5c06cd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:36.231450 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3937bfea-1485-46b5-a877-d5ab3c5c06cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:36.251523 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3937bfea-1485-46b5-a877-d5ab3c5c06cd None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:32:36.252946 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3937bfea-1485-46b5-a877-d5ab3c5c06cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:32:36.320239 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1103/2204] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:32:36.328117 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68837ad-0ada-4525-96c4-cc56248f4fab None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:36.328410 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68837ad-0ada-4525-96c4-cc56248f4fab None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:36.328692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68837ad-0ada-4525-96c4-cc56248f4fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:36.350066 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e68837ad-0ada-4525-96c4-cc56248f4fab None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:32:36.351302 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e68837ad-0ada-4525-96c4-cc56248f4fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:32:36.361108 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1102/2205] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:32:36.363365 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937bfea-1485-46b5-a877-d5ab3c5c06cd None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:32:36.373919 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3937bfea-1485-46b5-a877-d5ab3c5c06cd None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:32:36.378597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6ded1-7a9e-4c7f-bfe8-435dd299b51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:32:36.379213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6ded1-7a9e-4c7f-bfe8-435dd299b51c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:36.379494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6ded1-7a9e-4c7f-bfe8-435dd299b51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:36.379767 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6ded1-7a9e-4c7f-bfe8-435dd299b51c admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:36.381935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6ded1-7a9e-4c7f-bfe8-435dd299b51c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:32:36.382250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6ded1-7a9e-4c7f-bfe8-435dd299b51c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:32:36.383843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d6ded1-7a9e-4c7f-bfe8-435dd299b51c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:32:36.395845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1104/2206] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:32:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:32:36.400182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68837ad-0ada-4525-96c4-cc56248f4fab None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:32:36.409397 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68837ad-0ada-4525-96c4-cc56248f4fab None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:32:36.414679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c79e4-d8d2-41b3-bded-a8726cca5549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:32:36.415290 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c79e4-d8d2-41b3-bded-a8726cca5549 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:36.415570 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c79e4-d8d2-41b3-bded-a8726cca5549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:36.415846 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c79e4-d8d2-41b3-bded-a8726cca5549 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:36.416522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c79e4-d8d2-41b3-bded-a8726cca5549 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:32:36.418473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c79e4-d8d2-41b3-bded-a8726cca5549 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:32:36.423075 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:32:36.449423 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1103/2207] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:32:36 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:32:36.453661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6ded1-7a9e-4c7f-bfe8-435dd299b51c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:32:36.464500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6ded1-7a9e-4c7f-bfe8-435dd299b51c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:32:36.468148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:32:36.527860 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb7689-ea74-4efb-ad55-633fc1a5c860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:32:36.528493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb7689-ea74-4efb-ad55-633fc1a5c860 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:36.528780 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb7689-ea74-4efb-ad55-633fc1a5c860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:36.529054 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cb7689-ea74-4efb-ad55-633fc1a5c860 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:36.529745 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb7689-ea74-4efb-ad55-633fc1a5c860 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:32:36.531661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cb7689-ea74-4efb-ad55-633fc1a5c860 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:32:36.554519 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1105/2208] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:32:36 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:32:36.559328 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c79e4-d8d2-41b3-bded-a8726cca5549 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:32:36.570661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c79e4-d8d2-41b3-bded-a8726cca5549 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:32:36.575457 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2588fa-d6a5-4921-81f2-1136cc066fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:32:36.576078 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2588fa-d6a5-4921-81f2-1136cc066fc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:36.576380 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2588fa-d6a5-4921-81f2-1136cc066fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:36.576633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2588fa-d6a5-4921-81f2-1136cc066fc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:36.577395 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2588fa-d6a5-4921-81f2-1136cc066fc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:32:36.579052 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2588fa-d6a5-4921-81f2-1136cc066fc1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:32:36.590836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1104/2209] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:32:36.595405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb7689-ea74-4efb-ad55-633fc1a5c860 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:32:36.607833 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cb7689-ea74-4efb-ad55-633fc1a5c860 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:32:36.614260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4934c1e-93fe-47d4-9957-a849ab19d8cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:32:36.616372 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4934c1e-93fe-47d4-9957-a849ab19d8cf admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:36.617012 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4934c1e-93fe-47d4-9957-a849ab19d8cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:36.617012 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4934c1e-93fe-47d4-9957-a849ab19d8cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:36.618032 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4934c1e-93fe-47d4-9957-a849ab19d8cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:32:36.620182 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4934c1e-93fe-47d4-9957-a849ab19d8cf admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:32:36.639691 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1106/2210] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:32:36 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:32:36.640819 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2588fa-d6a5-4921-81f2-1136cc066fc1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:32:36.651671 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2588fa-d6a5-4921-81f2-1136cc066fc1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:32:36.656459 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693072c-84d1-4281-84ab-cbb8b88ea137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:32:36.657136 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693072c-84d1-4281-84ab-cbb8b88ea137 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:36.657421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693072c-84d1-4281-84ab-cbb8b88ea137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:36.657738 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693072c-84d1-4281-84ab-cbb8b88ea137 admin admin] PATH_INFO: `/v3/projects/862a3f47ebe842fb8c5fbd4ef08f0b5f/users/32eadc02f9224eb28303aab25506ef2a/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:36.674592 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693072c-84d1-4281-84ab-cbb8b88ea137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=862a3f47ebe842fb8c5fbd4ef08f0b5f, user_id=32eadc02f9224eb28303aab25506ef2a, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:32:36.677333 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693072c-84d1-4281-84ab-cbb8b88ea137 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:32:36.690582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1105/2211] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:32:36 2022] PUT /identity/v3/projects/862a3f47ebe842fb8c5fbd4ef08f0b5f/users/32eadc02f9224eb28303aab25506ef2a/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:32:36.696675 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819d970-95aa-4970-860e-5bb6a1845829 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:36.696969 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819d970-95aa-4970-860e-5bb6a1845829 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:36.697264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819d970-95aa-4970-860e-5bb6a1845829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:36.717730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8819d970-95aa-4970-860e-5bb6a1845829 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:32:36.718943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8819d970-95aa-4970-860e-5bb6a1845829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:32:36.781589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1107/2212] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:32:36.791170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693072c-84d1-4281-84ab-cbb8b88ea137 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:32:36.792901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4693072c-84d1-4281-84ab-cbb8b88ea137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:32:36.848954 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693072c-84d1-4281-84ab-cbb8b88ea137 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:32:36.852801 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca58d38-13f7-4c5e-a822-18c3113878c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:32:36.853413 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca58d38-13f7-4c5e-a822-18c3113878c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:36.853747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca58d38-13f7-4c5e-a822-18c3113878c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:36.854027 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca58d38-13f7-4c5e-a822-18c3113878c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:36.862980 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca58d38-13f7-4c5e-a822-18c3113878c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:32:36.864400 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca58d38-13f7-4c5e-a822-18c3113878c3 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:32:36.874161 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1106/2213] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:32:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:32:43.734850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ad4d5-abdd-4e7e-a1b8-5d077b5cae7e None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:43.735384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ad4d5-abdd-4e7e-a1b8-5d077b5cae7e None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:43.735384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ad4d5-abdd-4e7e-a1b8-5d077b5cae7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:43.769895 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-bd8ad4d5-abdd-4e7e-a1b8-5d077b5cae7e None None] MFA Rules not processed for user `32eadc02f9224eb28303aab25506ef2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:32:43.771444 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bd8ad4d5-abdd-4e7e-a1b8-5d077b5cae7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:32:43.869335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca58d38-13f7-4c5e-a822-18c3113878c3 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:32:43.870011 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1108/2214] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:32:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:32:43.880764 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca58d38-13f7-4c5e-a822-18c3113878c3 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:32:43.896359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75013059-7cab-49bb-b5f5-02a7d2954e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:32:43.897102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75013059-7cab-49bb-b5f5-02a7d2954e9d admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:43.897464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75013059-7cab-49bb-b5f5-02a7d2954e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:43.897831 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75013059-7cab-49bb-b5f5-02a7d2954e9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:43.899500 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75013059-7cab-49bb-b5f5-02a7d2954e9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:32:43.901955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75013059-7cab-49bb-b5f5-02a7d2954e9d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:32:43.939738 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1107/2215] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:32:43 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:32:43.944025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ad4d5-abdd-4e7e-a1b8-5d077b5cae7e None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:32:43.959524 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ad4d5-abdd-4e7e-a1b8-5d077b5cae7e None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:32:43.992517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:32:44.067237 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ab5b8-856f-4ebf-8e83-ebb314b0f5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:32:44.068462 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ab5b8-856f-4ebf-8e83-ebb314b0f5a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:44.068749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ab5b8-856f-4ebf-8e83-ebb314b0f5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:44.069026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531ab5b8-856f-4ebf-8e83-ebb314b0f5a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:44.069739 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ab5b8-856f-4ebf-8e83-ebb314b0f5a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:32:44.071705 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531ab5b8-856f-4ebf-8e83-ebb314b0f5a6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:32:44.113424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1109/2216] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:32:43 2022] POST /identity/v3/users => generated 295 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:32:44.115440 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75013059-7cab-49bb-b5f5-02a7d2954e9d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:32:44.133551 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75013059-7cab-49bb-b5f5-02a7d2954e9d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:32:44.154402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8138f-f2cd-48e1-bd6c-2391e63e36ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:32:44.155368 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8138f-f2cd-48e1-bd6c-2391e63e36ae admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:44.155651 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8138f-f2cd-48e1-bd6c-2391e63e36ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:44.155923 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab8138f-f2cd-48e1-bd6c-2391e63e36ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:44.157247 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8138f-f2cd-48e1-bd6c-2391e63e36ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:32:44.158750 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab8138f-f2cd-48e1-bd6c-2391e63e36ae admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:32:44.181463 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1108/2217] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:32:44.186140 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ab5b8-856f-4ebf-8e83-ebb314b0f5a6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:32:44.198606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531ab5b8-856f-4ebf-8e83-ebb314b0f5a6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:32:44.205448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f59373-c5e0-4130-a062-2a654f7ca9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:32:44.206264 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f59373-c5e0-4130-a062-2a654f7ca9d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:44.206558 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f59373-c5e0-4130-a062-2a654f7ca9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:44.206838 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f59373-c5e0-4130-a062-2a654f7ca9d1 admin admin] PATH_INFO: `/v3/projects/be22185cb3a1487787eaaa875566eebe/users/11e44c7d3e7e4d9c83dd92f4d3757212/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:44.226745 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f59373-c5e0-4130-a062-2a654f7ca9d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be22185cb3a1487787eaaa875566eebe, user_id=11e44c7d3e7e4d9c83dd92f4d3757212, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:32:44.229477 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f59373-c5e0-4130-a062-2a654f7ca9d1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:32:44.246956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1110/2218] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:32:44 2022] PUT /identity/v3/projects/be22185cb3a1487787eaaa875566eebe/users/11e44c7d3e7e4d9c83dd92f4d3757212/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:32:44.250670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8138f-f2cd-48e1-bd6c-2391e63e36ae admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:32:44.262853 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab8138f-f2cd-48e1-bd6c-2391e63e36ae admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:32:44.348649 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d0a12-baef-4e13-ab53-9530e601c453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:32:44.349837 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d0a12-baef-4e13-ab53-9530e601c453 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:44.350406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d0a12-baef-4e13-ab53-9530e601c453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:44.350688 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d0a12-baef-4e13-ab53-9530e601c453 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:44.351616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d0a12-baef-4e13-ab53-9530e601c453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:32:44.353331 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d0a12-baef-4e13-ab53-9530e601c453 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:32:44.363886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1109/2219] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:32:44 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:32:44.369677 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f59373-c5e0-4130-a062-2a654f7ca9d1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:32:44.380867 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f59373-c5e0-4130-a062-2a654f7ca9d1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:32:44.385765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42642483-79e1-487b-ac22-f6d703310407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:32:44.386442 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42642483-79e1-487b-ac22-f6d703310407 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:44.386729 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42642483-79e1-487b-ac22-f6d703310407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:44.387038 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42642483-79e1-487b-ac22-f6d703310407 admin admin] PATH_INFO: `/v3/projects/be22185cb3a1487787eaaa875566eebe/users/11e44c7d3e7e4d9c83dd92f4d3757212/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:44.389025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42642483-79e1-487b-ac22-f6d703310407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be22185cb3a1487787eaaa875566eebe, user_id=11e44c7d3e7e4d9c83dd92f4d3757212, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:32:44.391836 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42642483-79e1-487b-ac22-f6d703310407 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:32:44.402888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1111/2220] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:32:44 2022] PUT /identity/v3/projects/be22185cb3a1487787eaaa875566eebe/users/11e44c7d3e7e4d9c83dd92f4d3757212/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:32:50.426425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07250f-4186-462a-8cea-c2fe16a4591a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:50.427568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07250f-4186-462a-8cea-c2fe16a4591a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:50.428196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a07250f-4186-462a-8cea-c2fe16a4591a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:50.471188 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6a07250f-4186-462a-8cea-c2fe16a4591a None None] MFA Rules not processed for user `11e44c7d3e7e4d9c83dd92f4d3757212`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:32:50.472690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6a07250f-4186-462a-8cea-c2fe16a4591a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:32:50.478362 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:32:50.632392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1110/2221] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:32:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:32:50.652484 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42642483-79e1-487b-ac22-f6d703310407 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:32:50.654272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-42642483-79e1-487b-ac22-f6d703310407 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:32:50.779888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42642483-79e1-487b-ac22-f6d703310407 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:32:50.787348 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceea022-9cfb-4c5e-86ed-a01c71e15999 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:32:50.788164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceea022-9cfb-4c5e-86ed-a01c71e15999 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:50.788556 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceea022-9cfb-4c5e-86ed-a01c71e15999 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:50.789069 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceea022-9cfb-4c5e-86ed-a01c71e15999 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:50.855881 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceea022-9cfb-4c5e-86ed-a01c71e15999 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:32:50.857683 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceea022-9cfb-4c5e-86ed-a01c71e15999 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:32:50.869103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1112/2222] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:32:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:32:52.746678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073690fc-2a53-4f1b-8b0d-dab062002951 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:52.747346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073690fc-2a53-4f1b-8b0d-dab062002951 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:52.747346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073690fc-2a53-4f1b-8b0d-dab062002951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:52.791154 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-073690fc-2a53-4f1b-8b0d-dab062002951 None None] MFA Rules not processed for user `afbab63bf079403b8ef652422d581d25`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:32:52.794865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-073690fc-2a53-4f1b-8b0d-dab062002951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:32:52.899306 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1111/2223] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:32:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:32:55.609671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4dd2d-c273-48ce-b4e5-c16b1e16f802 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:55.610175 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4dd2d-c273-48ce-b4e5-c16b1e16f802 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:55.610352 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d4dd2d-c273-48ce-b4e5-c16b1e16f802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:55.619099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:32:55.640739 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-79d4dd2d-c273-48ce-b4e5-c16b1e16f802 None None] MFA Rules not processed for user `b7e13e8c76cb4757a42568112d16bd27`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:32:55.642556 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-79d4dd2d-c273-48ce-b4e5-c16b1e16f802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:32:55.724452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1113/2224] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:32:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:32:56.115844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43d204-4105-4d25-a50d-c58196ba9511 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:32:56.116312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43d204-4105-4d25-a50d-c58196ba9511 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:32:56.116597 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43d204-4105-4d25-a50d-c58196ba9511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:32:56.149431 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ce43d204-4105-4d25-a50d-c58196ba9511 None None] MFA Rules not processed for user `d38b517910894f7fa4351a2120b466d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:32:56.151698 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ce43d204-4105-4d25-a50d-c58196ba9511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:32:56.229570 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1112/2225] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:32:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:33:03.052864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4dd2d-c273-48ce-b4e5-c16b1e16f802 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:33:03.106210 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d4dd2d-c273-48ce-b4e5-c16b1e16f802 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:33:03.109768 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0d0e87-4a8f-4163-9e53-ed349cd22239 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:33:03.110381 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d0e87-4a8f-4163-9e53-ed349cd22239 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:33:03.110663 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d0e87-4a8f-4163-9e53-ed349cd22239 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:33:03.110934 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0d0e87-4a8f-4163-9e53-ed349cd22239 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:33:03.112658 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5c0d0e87-4a8f-4163-9e53-ed349cd22239 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:33:03.126512 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d0e87-4a8f-4163-9e53-ed349cd22239 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:33:03.127934 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0d0e87-4a8f-4163-9e53-ed349cd22239 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:33:03.139175 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1114/2226] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:33:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:33:04.120261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43d204-4105-4d25-a50d-c58196ba9511 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:33:04.130485 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43d204-4105-4d25-a50d-c58196ba9511 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:33:04.134070 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c551f9f-af34-4336-a17d-400af640b7ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:33:04.134750 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c551f9f-af34-4336-a17d-400af640b7ff service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:33:04.135093 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c551f9f-af34-4336-a17d-400af640b7ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:33:04.135430 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c551f9f-af34-4336-a17d-400af640b7ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:33:04.137232 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0c551f9f-af34-4336-a17d-400af640b7ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:33:04.152647 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c551f9f-af34-4336-a17d-400af640b7ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:33:04.154124 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c551f9f-af34-4336-a17d-400af640b7ff service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:33:04.168051 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1113/2227] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:33:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:33:09.116885 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc756246-3ac3-4fa7-917a-ee3d7884e996 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:33:09.117416 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc756246-3ac3-4fa7-917a-ee3d7884e996 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:33:09.117416 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc756246-3ac3-4fa7-917a-ee3d7884e996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:33:09.141772 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-fc756246-3ac3-4fa7-917a-ee3d7884e996 None None] MFA Rules not processed for user `32eadc02f9224eb28303aab25506ef2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:33:09.142981 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fc756246-3ac3-4fa7-917a-ee3d7884e996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:33:09.152112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1115/2228] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:33:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:33:14.162626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=83 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:33:19.137040 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=52 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:33:20.478961 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c551f9f-af34-4336-a17d-400af640b7ff service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:33:20.480333 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0c551f9f-af34-4336-a17d-400af640b7ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:33:20.538053 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c551f9f-af34-4336-a17d-400af640b7ff service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:33:20.541724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a804c-6d57-4081-84aa-49edd9a4cac9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:33:20.542334 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a804c-6d57-4081-84aa-49edd9a4cac9 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:33:20.542615 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a804c-6d57-4081-84aa-49edd9a4cac9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:33:20.542888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a804c-6d57-4081-84aa-49edd9a4cac9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:33:20.544638 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c22a804c-6d57-4081-84aa-49edd9a4cac9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:33:20.603327 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a804c-6d57-4081-84aa-49edd9a4cac9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:33:20.604764 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a804c-6d57-4081-84aa-49edd9a4cac9 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:33:20.614897 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1114/2229] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:33:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:33:27.668357 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6219ee7-a93b-4918-852e-bd18cb7c2c6d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:33:27.668887 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6219ee7-a93b-4918-852e-bd18cb7c2c6d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:33:27.668887 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6219ee7-a93b-4918-852e-bd18cb7c2c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:33:27.688981 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f6219ee7-a93b-4918-852e-bd18cb7c2c6d None None] MFA Rules not processed for user `7476eccb0a6f4195936bec75f7f80c7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:33:27.690211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f6219ee7-a93b-4918-852e-bd18cb7c2c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:33:27.697082 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1116/2230] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:33:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:33:28.005433 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a804c-6d57-4081-84aa-49edd9a4cac9 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:33:28.014479 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a804c-6d57-4081-84aa-49edd9a4cac9 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:33:28.017994 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c69a0-2fb6-4da4-a04f-08f42d65848c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:33:28.018594 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c69a0-2fb6-4da4-a04f-08f42d65848c service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:33:28.018871 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c69a0-2fb6-4da4-a04f-08f42d65848c service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:33:28.019141 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c69a0-2fb6-4da4-a04f-08f42d65848c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:33:28.021594 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5f7c69a0-2fb6-4da4-a04f-08f42d65848c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:33:28.030919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c69a0-2fb6-4da4-a04f-08f42d65848c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:33:28.032355 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c69a0-2fb6-4da4-a04f-08f42d65848c service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:33:28.040873 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1115/2231] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:33:28 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:33:28.346319 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6219ee7-a93b-4918-852e-bd18cb7c2c6d None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:33:28.347892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f6219ee7-a93b-4918-852e-bd18cb7c2c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:33:28.422129 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6219ee7-a93b-4918-852e-bd18cb7c2c6d None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:33:28.427545 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81e769-2c31-4f91-ab89-c68632d54ab4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe003cd7bfbe4b92b1d64d300c3e7d8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:33:28.428159 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81e769-2c31-4f91-ab89-c68632d54ab4 service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:33:28.428441 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81e769-2c31-4f91-ab89-c68632d54ab4 service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:33:28.428716 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81e769-2c31-4f91-ab89-c68632d54ab4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:33:28.430486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2c81e769-2c31-4f91-ab89-c68632d54ab4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:33:28.442082 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81e769-2c31-4f91-ab89-c68632d54ab4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:33:28.443643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81e769-2c31-4f91-ab89-c68632d54ab4 service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:33:28.453339 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1117/2232] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:33:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:33:28.945576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c69a0-2fb6-4da4-a04f-08f42d65848c service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:33:28.954245 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c69a0-2fb6-4da4-a04f-08f42d65848c service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:33:28.959495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e627b-0172-43cb-9fe6-57d67f3b6981 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:33:28.960114 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e627b-0172-43cb-9fe6-57d67f3b6981 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:33:28.960495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e627b-0172-43cb-9fe6-57d67f3b6981 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:33:28.960771 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e627b-0172-43cb-9fe6-57d67f3b6981 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:33:28.970461 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e627b-0172-43cb-9fe6-57d67f3b6981 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:33:28.971870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e627b-0172-43cb-9fe6-57d67f3b6981 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:33:28.982552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1116/2233] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:33:28 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:33:37.591531 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81e769-2c31-4f91-ab89-c68632d54ab4 service placement] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:33:37.593046 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2c81e769-2c31-4f91-ab89-c68632d54ab4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:33:37.604490 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81e769-2c31-4f91-ab89-c68632d54ab4 service placement] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:33:37.608129 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c3c16-26a5-4ec1-8373-3f0456eb27b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:33:37.608762 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c3c16-26a5-4ec1-8373-3f0456eb27b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:33:37.609044 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c3c16-26a5-4ec1-8373-3f0456eb27b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:33:37.609319 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c3c16-26a5-4ec1-8373-3f0456eb27b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:33:37.611050 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-826c3c16-26a5-4ec1-8373-3f0456eb27b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:33:37.620954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c3c16-26a5-4ec1-8373-3f0456eb27b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:33:37.622344 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c3c16-26a5-4ec1-8373-3f0456eb27b6 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:33:37.631312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1118/2234] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:33:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:33:38.967204 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e627b-0172-43cb-9fe6-57d67f3b6981 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:33:38.976699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e627b-0172-43cb-9fe6-57d67f3b6981 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:33:38.980270 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276a232-22bd-4de8-913e-1f27a9115e04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:33:38.980872 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276a232-22bd-4de8-913e-1f27a9115e04 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:33:38.981154 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276a232-22bd-4de8-913e-1f27a9115e04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:33:38.981429 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276a232-22bd-4de8-913e-1f27a9115e04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:33:38.983190 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3276a232-22bd-4de8-913e-1f27a9115e04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:33:38.994807 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276a232-22bd-4de8-913e-1f27a9115e04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:33:38.996241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276a232-22bd-4de8-913e-1f27a9115e04 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:33:39.006424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1117/2235] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:33:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:33:41.421989 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c3c16-26a5-4ec1-8373-3f0456eb27b6 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:33:41.423391 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-826c3c16-26a5-4ec1-8373-3f0456eb27b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:33:41.440619 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c3c16-26a5-4ec1-8373-3f0456eb27b6 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:33:41.447477 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744a9ae-7edc-4275-85ec-59b8513b8166 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:33:41.448183 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744a9ae-7edc-4275-85ec-59b8513b8166 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:33:41.449751 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744a9ae-7edc-4275-85ec-59b8513b8166 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:33:41.449751 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744a9ae-7edc-4275-85ec-59b8513b8166 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:33:41.450717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0744a9ae-7edc-4275-85ec-59b8513b8166 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:33:41.463900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744a9ae-7edc-4275-85ec-59b8513b8166 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:33:41.465503 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744a9ae-7edc-4275-85ec-59b8513b8166 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:33:41.483294 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1119/2236] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:33:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:33:46.728703 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276a232-22bd-4de8-913e-1f27a9115e04 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:33:46.735886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=67 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:33:46.758036 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276a232-22bd-4de8-913e-1f27a9115e04 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:33:46.758036 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064b704-5aa0-4d1f-b4b8-3c4b77701ef8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:33:46.758036 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064b704-5aa0-4d1f-b4b8-3c4b77701ef8 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:33:46.758036 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064b704-5aa0-4d1f-b4b8-3c4b77701ef8 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:33:46.758592 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a064b704-5aa0-4d1f-b4b8-3c4b77701ef8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:33:46.758592 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a064b704-5aa0-4d1f-b4b8-3c4b77701ef8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:33:46.773065 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064b704-5aa0-4d1f-b4b8-3c4b77701ef8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:33:46.774873 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a064b704-5aa0-4d1f-b4b8-3c4b77701ef8 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:33:46.792017 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1118/2237] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:33:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:33:51.473438 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=47 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:33:55.389142 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744a9ae-7edc-4275-85ec-59b8513b8166 service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:33:55.400157 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744a9ae-7edc-4275-85ec-59b8513b8166 service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:33:55.405582 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c9058-efcf-4f35-a2ce-cd292d03c2b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:33:55.406193 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c9058-efcf-4f35-a2ce-cd292d03c2b2 service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:33:55.406471 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c9058-efcf-4f35-a2ce-cd292d03c2b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:33:55.406743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c9058-efcf-4f35-a2ce-cd292d03c2b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:33:55.408564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-788c9058-efcf-4f35-a2ce-cd292d03c2b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:33:55.419580 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c9058-efcf-4f35-a2ce-cd292d03c2b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:33:55.421145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c9058-efcf-4f35-a2ce-cd292d03c2b2 service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:33:55.430733 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1120/2238] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:33:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:33:56.781691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:33:58.011495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064b704-5aa0-4d1f-b4b8-3c4b77701ef8 service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:33:58.034191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a064b704-5aa0-4d1f-b4b8-3c4b77701ef8 service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:33:58.041186 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5c689-6294-44a5-9378-104285cd8410 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:33:58.042054 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5c689-6294-44a5-9378-104285cd8410 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:33:58.042437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5c689-6294-44a5-9378-104285cd8410 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:33:58.042807 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b5c689-6294-44a5-9378-104285cd8410 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:33:58.044706 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-44b5c689-6294-44a5-9378-104285cd8410 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:33:58.056653 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b5c689-6294-44a5-9378-104285cd8410 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:33:58.058612 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b5c689-6294-44a5-9378-104285cd8410 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:33:58.068845 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1119/2239] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:33:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:34:01.440613 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c9058-efcf-4f35-a2ce-cd292d03c2b2 service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:01.449808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c9058-efcf-4f35-a2ce-cd292d03c2b2 service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:01.453356 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21473f-286e-4dfe-911d-b74e3bff3fc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:01.453998 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21473f-286e-4dfe-911d-b74e3bff3fc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:01.454277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21473f-286e-4dfe-911d-b74e3bff3fc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:01.454548 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c21473f-286e-4dfe-911d-b74e3bff3fc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:01.456360 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4c21473f-286e-4dfe-911d-b74e3bff3fc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:01.544691 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21473f-286e-4dfe-911d-b74e3bff3fc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:01.546440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c21473f-286e-4dfe-911d-b74e3bff3fc7 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:01.550165 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5c689-6294-44a5-9378-104285cd8410 service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:01.566415 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1121/2240] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:34:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:34:01.567380 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b5c689-6294-44a5-9378-104285cd8410 service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:01.574207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2af3c-56c4-4448-9fd9-d77f333a759b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:01.575201 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2af3c-56c4-4448-9fd9-d77f333a759b service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:01.575637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2af3c-56c4-4448-9fd9-d77f333a759b service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:01.576065 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2af3c-56c4-4448-9fd9-d77f333a759b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:01.578148 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-05e2af3c-56c4-4448-9fd9-d77f333a759b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:01.592551 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2af3c-56c4-4448-9fd9-d77f333a759b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:01.595342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2af3c-56c4-4448-9fd9-d77f333a759b service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:01.607931 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1120/2241] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:34:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:34:05.076753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21473f-286e-4dfe-911d-b74e3bff3fc7 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:05.078537 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4c21473f-286e-4dfe-911d-b74e3bff3fc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:05.092893 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c21473f-286e-4dfe-911d-b74e3bff3fc7 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:05.097517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5371579-3b29-473c-9a27-aa9b5699a1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:05.098221 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5371579-3b29-473c-9a27-aa9b5699a1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:05.098503 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5371579-3b29-473c-9a27-aa9b5699a1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:05.098779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5371579-3b29-473c-9a27-aa9b5699a1fc admin admin] PATH_INFO: `/v3/users/afbab63bf079403b8ef652422d581d25` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:05.100433 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5371579-3b29-473c-9a27-aa9b5699a1fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=afbab63bf079403b8ef652422d581d25)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:05.102920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5371579-3b29-473c-9a27-aa9b5699a1fc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:05.155111 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a5371579-3b29-473c-9a27-aa9b5699a1fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afbab63bf079403b8ef652422d581d25'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:05.164521 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a5371579-3b29-473c-9a27-aa9b5699a1fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afbab63bf079403b8ef652422d581d25'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:05.172801 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a5371579-3b29-473c-9a27-aa9b5699a1fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afbab63bf079403b8ef652422d581d25'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:05.173936 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a5371579-3b29-473c-9a27-aa9b5699a1fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afbab63bf079403b8ef652422d581d25'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:05.188410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a5371579-3b29-473c-9a27-aa9b5699a1fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afbab63bf079403b8ef652422d581d25'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:05.192904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1122/2242] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:34:05 2022] DELETE /identity/v3/users/afbab63bf079403b8ef652422d581d25 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:05.375726 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2af3c-56c4-4448-9fd9-d77f333a759b service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:05.390257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2af3c-56c4-4448-9fd9-d77f333a759b service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:05.447762 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06b1265-4f05-4d4a-9063-b5e707506d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:05.448330 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b1265-4f05-4d4a-9063-b5e707506d3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:05.448615 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b1265-4f05-4d4a-9063-b5e707506d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:05.448894 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06b1265-4f05-4d4a-9063-b5e707506d3f admin admin] PATH_INFO: `/v3/projects/567bd0fe29da4db5a0f08c8b5cc81f9c` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:05.450094 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b1265-4f05-4d4a-9063-b5e707506d3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=567bd0fe29da4db5a0f08c8b5cc81f9c)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:05.452026 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06b1265-4f05-4d4a-9063-b5e707506d3f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:05.499286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c06b1265-4f05-4d4a-9063-b5e707506d3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '567bd0fe29da4db5a0f08c8b5cc81f9c', 'request_id': 'req-c06b1265-4f05-4d4a-9063-b5e707506d3f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:05.505498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c06b1265-4f05-4d4a-9063-b5e707506d3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '567bd0fe29da4db5a0f08c8b5cc81f9c', 'request_id': 'req-c06b1265-4f05-4d4a-9063-b5e707506d3f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:05.511243 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c06b1265-4f05-4d4a-9063-b5e707506d3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '567bd0fe29da4db5a0f08c8b5cc81f9c', 'request_id': 'req-c06b1265-4f05-4d4a-9063-b5e707506d3f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:05.525046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c06b1265-4f05-4d4a-9063-b5e707506d3f admin admin] The token cache is being invalidate because project 567bd0fe29da4db5a0f08c8b5cc81f9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:34:05.525341 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c06b1265-4f05-4d4a-9063-b5e707506d3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:05.526046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c06b1265-4f05-4d4a-9063-b5e707506d3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:05.527675 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1121/2243] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:34:05 2022] DELETE /identity/v3/projects/567bd0fe29da4db5a0f08c8b5cc81f9c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:05.532686 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593bfe2-5442-424c-b959-6d8b8f81f72b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:05.533041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593bfe2-5442-424c-b959-6d8b8f81f72b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:05.533325 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593bfe2-5442-424c-b959-6d8b8f81f72b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:05.554050 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-0593bfe2-5442-424c-b959-6d8b8f81f72b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:05.555253 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0593bfe2-5442-424c-b959-6d8b8f81f72b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:05.604795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:05.618817 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1123/2244] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:05.626900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30272c7-4bea-49c3-992f-5e50a93ab8fa None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:05.627192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30272c7-4bea-49c3-992f-5e50a93ab8fa None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:05.627478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30272c7-4bea-49c3-992f-5e50a93ab8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:05.647839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f30272c7-4bea-49c3-992f-5e50a93ab8fa None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:05.649017 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f30272c7-4bea-49c3-992f-5e50a93ab8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:05.656914 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1122/2245] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:05.661104 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593bfe2-5442-424c-b959-6d8b8f81f72b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:05.671484 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593bfe2-5442-424c-b959-6d8b8f81f72b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:05.676031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97b3cf-383b-4bfb-b89d-ad18299b82bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:05.676640 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97b3cf-383b-4bfb-b89d-ad18299b82bf admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:05.676925 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97b3cf-383b-4bfb-b89d-ad18299b82bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:05.677203 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a97b3cf-383b-4bfb-b89d-ad18299b82bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:05.677995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97b3cf-383b-4bfb-b89d-ad18299b82bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:34:05.678385 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97b3cf-383b-4bfb-b89d-ad18299b82bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:05.679996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a97b3cf-383b-4bfb-b89d-ad18299b82bf admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:05.691404 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1124/2246] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:34:05.696250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30272c7-4bea-49c3-992f-5e50a93ab8fa None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:05.705554 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30272c7-4bea-49c3-992f-5e50a93ab8fa None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:05.710747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5223c-4759-4021-8bf0-1c26dd2d3bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:05.711342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5223c-4759-4021-8bf0-1c26dd2d3bb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:05.711630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5223c-4759-4021-8bf0-1c26dd2d3bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:05.711906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5223c-4759-4021-8bf0-1c26dd2d3bb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:05.712558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5223c-4759-4021-8bf0-1c26dd2d3bb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:05.714502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5223c-4759-4021-8bf0-1c26dd2d3bb7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:05.741212 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1123/2247] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:34:05 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:34:05.745271 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97b3cf-383b-4bfb-b89d-ad18299b82bf admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:05.755737 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a97b3cf-383b-4bfb-b89d-ad18299b82bf admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:05.811364 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eca543-2686-4bdc-b577-2cfb09824e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:05.811974 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eca543-2686-4bdc-b577-2cfb09824e00 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:05.812253 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eca543-2686-4bdc-b577-2cfb09824e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:05.812525 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eca543-2686-4bdc-b577-2cfb09824e00 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:05.813180 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eca543-2686-4bdc-b577-2cfb09824e00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:05.815146 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eca543-2686-4bdc-b577-2cfb09824e00 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:05.842295 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1125/2248] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:34:05 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:34:05.846276 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5223c-4759-4021-8bf0-1c26dd2d3bb7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:05.857380 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5223c-4759-4021-8bf0-1c26dd2d3bb7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:05.862746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc7ad5-6b06-4c9a-be34-41e7107f8622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:05.863518 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc7ad5-6b06-4c9a-be34-41e7107f8622 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:05.863886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc7ad5-6b06-4c9a-be34-41e7107f8622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:05.864302 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc7ad5-6b06-4c9a-be34-41e7107f8622 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:05.865066 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc7ad5-6b06-4c9a-be34-41e7107f8622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:05.867234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc7ad5-6b06-4c9a-be34-41e7107f8622 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:05.878089 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1124/2249] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:34:05.883177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eca543-2686-4bdc-b577-2cfb09824e00 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:05.894514 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eca543-2686-4bdc-b577-2cfb09824e00 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:05.907719 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917dd179-0b85-4362-bcc7-df823099b640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:05.908347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dd179-0b85-4362-bcc7-df823099b640 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:05.908629 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dd179-0b85-4362-bcc7-df823099b640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:05.908903 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dd179-0b85-4362-bcc7-df823099b640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:05.909656 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917dd179-0b85-4362-bcc7-df823099b640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:05.911262 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917dd179-0b85-4362-bcc7-df823099b640 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:05.923111 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1126/2250] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:34:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:34:05.927294 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc7ad5-6b06-4c9a-be34-41e7107f8622 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:05.937902 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc7ad5-6b06-4c9a-be34-41e7107f8622 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:05.942508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda17e6e-4874-457e-9f1b-135c3138df9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:05.943185 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda17e6e-4874-457e-9f1b-135c3138df9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:05.943448 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda17e6e-4874-457e-9f1b-135c3138df9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:05.944270 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda17e6e-4874-457e-9f1b-135c3138df9a admin admin] PATH_INFO: `/v3/projects/f76093ef3ffa4de0a127a3b7de378a0f/users/59b79255482a4f1b992b6cd764923389/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:05.961301 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda17e6e-4874-457e-9f1b-135c3138df9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f76093ef3ffa4de0a127a3b7de378a0f, user_id=59b79255482a4f1b992b6cd764923389, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:05.964069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda17e6e-4874-457e-9f1b-135c3138df9a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:05.974178 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:05.983171 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1125/2251] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:34:05 2022] PUT /identity/v3/projects/f76093ef3ffa4de0a127a3b7de378a0f/users/59b79255482a4f1b992b6cd764923389/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:05.994487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b3732-1a31-4c81-9798-75821326a75b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:05.995106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b3732-1a31-4c81-9798-75821326a75b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:05.995420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26b3732-1a31-4c81-9798-75821326a75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:06.031187 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b26b3732-1a31-4c81-9798-75821326a75b None None] MFA Rules not processed for user `59b79255482a4f1b992b6cd764923389`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:06.032444 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b26b3732-1a31-4c81-9798-75821326a75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:06.101555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1127/2252] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:06.203906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda17e6e-4874-457e-9f1b-135c3138df9a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:06.205802 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bda17e6e-4874-457e-9f1b-135c3138df9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:06.279255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda17e6e-4874-457e-9f1b-135c3138df9a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:06.284663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d060e6-4f99-43fd-80a1-b9c3913937e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:06.285274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d060e6-4f99-43fd-80a1-b9c3913937e9 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:06.285590 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d060e6-4f99-43fd-80a1-b9c3913937e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:06.285867 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d060e6-4f99-43fd-80a1-b9c3913937e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:06.294303 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d060e6-4f99-43fd-80a1-b9c3913937e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:06.296014 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d060e6-4f99-43fd-80a1-b9c3913937e9 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:06.304978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1126/2253] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:34:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:34:06.506547 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b3732-1a31-4c81-9798-75821326a75b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:06.554180 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:06.569927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26b3732-1a31-4c81-9798-75821326a75b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:06.574370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7df75-28d0-405d-8b59-048391f0f555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:06.574998 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7df75-28d0-405d-8b59-048391f0f555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:06.575277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7df75-28d0-405d-8b59-048391f0f555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:06.575550 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e7df75-28d0-405d-8b59-048391f0f555 admin admin] PATH_INFO: `/v3/users/59b79255482a4f1b992b6cd764923389` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:06.576692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7df75-28d0-405d-8b59-048391f0f555 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59b79255482a4f1b992b6cd764923389)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:06.578646 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e7df75-28d0-405d-8b59-048391f0f555 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:06.623227 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e7e7df75-28d0-405d-8b59-048391f0f555 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59b79255482a4f1b992b6cd764923389'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:06.630584 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e7e7df75-28d0-405d-8b59-048391f0f555 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59b79255482a4f1b992b6cd764923389'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:06.638431 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e7e7df75-28d0-405d-8b59-048391f0f555 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59b79255482a4f1b992b6cd764923389'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:06.639057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e7e7df75-28d0-405d-8b59-048391f0f555 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59b79255482a4f1b992b6cd764923389'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:06.651179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e7e7df75-28d0-405d-8b59-048391f0f555 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59b79255482a4f1b992b6cd764923389'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:06.657221 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1128/2254] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:34:06 2022] DELETE /identity/v3/users/59b79255482a4f1b992b6cd764923389 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:06.661722 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0629f-7e89-4e27-93b6-f5193ff42cfd None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:06.662248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0629f-7e89-4e27-93b6-f5193ff42cfd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:06.662363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0629f-7e89-4e27-93b6-f5193ff42cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:06.682802 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-94f0629f-7e89-4e27-93b6-f5193ff42cfd None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:06.684323 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-94f0629f-7e89-4e27-93b6-f5193ff42cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:06.745107 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1127/2255] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:06.754957 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7df75-28d0-405d-8b59-048391f0f555 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:06.757414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e7e7df75-28d0-405d-8b59-048391f0f555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:06.814395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e7df75-28d0-405d-8b59-048391f0f555 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:06.818059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817a053-eae1-41c4-b830-79810a1a043e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:06.819428 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817a053-eae1-41c4-b830-79810a1a043e service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:06.819714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817a053-eae1-41c4-b830-79810a1a043e service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:06.819990 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817a053-eae1-41c4-b830-79810a1a043e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:06.828818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817a053-eae1-41c4-b830-79810a1a043e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:06.830258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817a053-eae1-41c4-b830-79810a1a043e service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:06.839581 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1129/2256] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:34:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:34:07.131182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0629f-7e89-4e27-93b6-f5193ff42cfd None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:07.132653 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-94f0629f-7e89-4e27-93b6-f5193ff42cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:07.145027 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0629f-7e89-4e27-93b6-f5193ff42cfd None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:07.150464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e821593-13a8-438a-960a-13c4d894b4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:07.151115 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e821593-13a8-438a-960a-13c4d894b4a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:07.151394 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e821593-13a8-438a-960a-13c4d894b4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:07.151786 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e821593-13a8-438a-960a-13c4d894b4a8 admin admin] PATH_INFO: `/v3/projects/f76093ef3ffa4de0a127a3b7de378a0f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:07.153092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e821593-13a8-438a-960a-13c4d894b4a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f76093ef3ffa4de0a127a3b7de378a0f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:07.155048 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e821593-13a8-438a-960a-13c4d894b4a8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:07.203237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6e821593-13a8-438a-960a-13c4d894b4a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f76093ef3ffa4de0a127a3b7de378a0f', 'request_id': 'req-6e821593-13a8-438a-960a-13c4d894b4a8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:07.210000 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6e821593-13a8-438a-960a-13c4d894b4a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f76093ef3ffa4de0a127a3b7de378a0f', 'request_id': 'req-6e821593-13a8-438a-960a-13c4d894b4a8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:07.215522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6e821593-13a8-438a-960a-13c4d894b4a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f76093ef3ffa4de0a127a3b7de378a0f', 'request_id': 'req-6e821593-13a8-438a-960a-13c4d894b4a8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:07.228812 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6e821593-13a8-438a-960a-13c4d894b4a8 admin admin] The token cache is being invalidate because project f76093ef3ffa4de0a127a3b7de378a0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:34:07.229103 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6e821593-13a8-438a-960a-13c4d894b4a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:07.229752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6e821593-13a8-438a-960a-13c4d894b4a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:07.231293 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1128/2257] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:34:07 2022] DELETE /identity/v3/projects/f76093ef3ffa4de0a127a3b7de378a0f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:07.236220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1696f7-b9bc-4e54-8c2e-65f0add38d00 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:07.237136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1696f7-b9bc-4e54-8c2e-65f0add38d00 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:07.237496 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1696f7-b9bc-4e54-8c2e-65f0add38d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:07.262050 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-0e1696f7-b9bc-4e54-8c2e-65f0add38d00 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:07.263411 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0e1696f7-b9bc-4e54-8c2e-65f0add38d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:07.340532 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1130/2258] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:07.343670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cfa96-3a67-4e5d-9a61-8842ee4c6f38 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:07.343839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cfa96-3a67-4e5d-9a61-8842ee4c6f38 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:07.344251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2cfa96-3a67-4e5d-9a61-8842ee4c6f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:07.383197 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-bd2cfa96-3a67-4e5d-9a61-8842ee4c6f38 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:07.384977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bd2cfa96-3a67-4e5d-9a61-8842ee4c6f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:07.398846 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1129/2259] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:07.403021 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1696f7-b9bc-4e54-8c2e-65f0add38d00 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:07.416083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1696f7-b9bc-4e54-8c2e-65f0add38d00 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:07.423871 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad914c1-8047-409e-bde2-3c61fb8ae7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:07.424744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad914c1-8047-409e-bde2-3c61fb8ae7d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:07.424993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad914c1-8047-409e-bde2-3c61fb8ae7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:07.425313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad914c1-8047-409e-bde2-3c61fb8ae7d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:07.426393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad914c1-8047-409e-bde2-3c61fb8ae7d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:34:07.426699 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad914c1-8047-409e-bde2-3c61fb8ae7d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:07.428765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad914c1-8047-409e-bde2-3c61fb8ae7d2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:07.436326 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:07.447357 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1131/2260] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:34:07.448526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2cfa96-3a67-4e5d-9a61-8842ee4c6f38 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:07.457372 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:07.479730 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2cfa96-3a67-4e5d-9a61-8842ee4c6f38 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:07.504414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd6b48-1158-4236-8aee-abd5e5751604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:07.504900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd6b48-1158-4236-8aee-abd5e5751604 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:07.505194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd6b48-1158-4236-8aee-abd5e5751604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:07.505451 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd6b48-1158-4236-8aee-abd5e5751604 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:07.506667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd6b48-1158-4236-8aee-abd5e5751604 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:07.509363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd6b48-1158-4236-8aee-abd5e5751604 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:07.538539 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1130/2261] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:34:07 2022] POST /identity/v3/projects => generated 401 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:34:07.543085 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad914c1-8047-409e-bde2-3c61fb8ae7d2 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:07.554192 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad914c1-8047-409e-bde2-3c61fb8ae7d2 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:07.611025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d4456-e9a5-41d5-974f-b785a972d843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:07.611633 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d4456-e9a5-41d5-974f-b785a972d843 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:07.611913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d4456-e9a5-41d5-974f-b785a972d843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:07.612192 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d4456-e9a5-41d5-974f-b785a972d843 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:07.612854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d4456-e9a5-41d5-974f-b785a972d843 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:07.614782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d4456-e9a5-41d5-974f-b785a972d843 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:07.635439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1132/2262] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:34:07 2022] POST /identity/v3/users => generated 301 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:34:07.639632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd6b48-1158-4236-8aee-abd5e5751604 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:07.650953 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd6b48-1158-4236-8aee-abd5e5751604 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:07.655518 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a9284-f185-4402-87fa-39383d8a192e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:07.656117 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a9284-f185-4402-87fa-39383d8a192e admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:07.656395 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a9284-f185-4402-87fa-39383d8a192e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:07.656668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9a9284-f185-4402-87fa-39383d8a192e admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:07.657369 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a9284-f185-4402-87fa-39383d8a192e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:07.658982 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9a9284-f185-4402-87fa-39383d8a192e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:07.670037 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1131/2263] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:34:07.674266 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d4456-e9a5-41d5-974f-b785a972d843 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:07.684823 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d4456-e9a5-41d5-974f-b785a972d843 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:07.689445 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d218ee43-27c5-4171-a569-0ac3932f572d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:07.690096 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218ee43-27c5-4171-a569-0ac3932f572d admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:07.690382 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218ee43-27c5-4171-a569-0ac3932f572d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:07.690661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d218ee43-27c5-4171-a569-0ac3932f572d admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:07.691408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218ee43-27c5-4171-a569-0ac3932f572d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:07.692988 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d218ee43-27c5-4171-a569-0ac3932f572d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:07.703871 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1133/2264] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:34:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:34:07.707819 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a9284-f185-4402-87fa-39383d8a192e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:07.718539 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9a9284-f185-4402-87fa-39383d8a192e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:07.723159 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4587b-cba3-4bd7-ba27-f01dead4ab6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:07.723813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4587b-cba3-4bd7-ba27-f01dead4ab6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:07.724102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4587b-cba3-4bd7-ba27-f01dead4ab6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:07.724379 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf4587b-cba3-4bd7-ba27-f01dead4ab6f admin admin] PATH_INFO: `/v3/projects/33e02f250f1f4531b44fa855799ee098/users/61813b61fa1a4519b57dac3c8c758982/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:07.739534 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf4587b-cba3-4bd7-ba27-f01dead4ab6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=33e02f250f1f4531b44fa855799ee098, user_id=61813b61fa1a4519b57dac3c8c758982, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:07.742199 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf4587b-cba3-4bd7-ba27-f01dead4ab6f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:07.755915 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1132/2265] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:34:07 2022] PUT /identity/v3/projects/33e02f250f1f4531b44fa855799ee098/users/61813b61fa1a4519b57dac3c8c758982/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:07.766559 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968664f-ed29-4aa8-89fd-b83627e0a6dc None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:07.767007 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968664f-ed29-4aa8-89fd-b83627e0a6dc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:07.767460 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968664f-ed29-4aa8-89fd-b83627e0a6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:07.796810 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c968664f-ed29-4aa8-89fd-b83627e0a6dc None None] MFA Rules not processed for user `61813b61fa1a4519b57dac3c8c758982`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:07.798137 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c968664f-ed29-4aa8-89fd-b83627e0a6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:07.866370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1134/2266] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:07 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:34:07.872151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4587b-cba3-4bd7-ba27-f01dead4ab6f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:07.938021 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf4587b-cba3-4bd7-ba27-f01dead4ab6f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:07.942655 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47fbf1-cb90-41b7-9a65-54ded91e277a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:07.943274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47fbf1-cb90-41b7-9a65-54ded91e277a admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:07.944193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47fbf1-cb90-41b7-9a65-54ded91e277a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:07.944670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b47fbf1-cb90-41b7-9a65-54ded91e277a admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:07.945706 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47fbf1-cb90-41b7-9a65-54ded91e277a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:07.947666 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b47fbf1-cb90-41b7-9a65-54ded91e277a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:07.979605 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1133/2267] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:34:07 2022] POST /identity/v3/projects => generated 399 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:34:07.983969 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968664f-ed29-4aa8-89fd-b83627e0a6dc None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:07.995668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968664f-ed29-4aa8-89fd-b83627e0a6dc None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:08.058200 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540bd6b-f2d2-4fa0-a368-6ae408f5f728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:08.058994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540bd6b-f2d2-4fa0-a368-6ae408f5f728 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:08.059336 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540bd6b-f2d2-4fa0-a368-6ae408f5f728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:08.059636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6540bd6b-f2d2-4fa0-a368-6ae408f5f728 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:08.060453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540bd6b-f2d2-4fa0-a368-6ae408f5f728 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:08.062540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6540bd6b-f2d2-4fa0-a368-6ae408f5f728 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:08.075327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:08.092319 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1135/2268] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:34:07 2022] POST /identity/v3/users => generated 306 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:34:08.096101 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47fbf1-cb90-41b7-9a65-54ded91e277a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:08.107985 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b47fbf1-cb90-41b7-9a65-54ded91e277a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:08.112818 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b438b-c583-476e-8dc8-7b837f8208b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:08.113642 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b438b-c583-476e-8dc8-7b837f8208b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:08.113953 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b438b-c583-476e-8dc8-7b837f8208b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:08.114221 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b438b-c583-476e-8dc8-7b837f8208b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:08.115207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b438b-c583-476e-8dc8-7b837f8208b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:08.117130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b438b-c583-476e-8dc8-7b837f8208b3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:08.130291 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1134/2269] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:34:08.134971 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540bd6b-f2d2-4fa0-a368-6ae408f5f728 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:08.146436 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6540bd6b-f2d2-4fa0-a368-6ae408f5f728 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:08.153972 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c87bd4-9b46-4455-959d-371fa0631f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:08.155061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c87bd4-9b46-4455-959d-371fa0631f93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:08.155377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c87bd4-9b46-4455-959d-371fa0631f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:08.155676 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c87bd4-9b46-4455-959d-371fa0631f93 admin admin] PATH_INFO: `/v3/projects/1e9f02f94a10489abd3ab38e919d9b55/users/7836268aec65433bb23ff975acb8b5c3/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:08.171438 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c87bd4-9b46-4455-959d-371fa0631f93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e9f02f94a10489abd3ab38e919d9b55, user_id=7836268aec65433bb23ff975acb8b5c3, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:08.174384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c87bd4-9b46-4455-959d-371fa0631f93 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:08.188190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1136/2270] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:34:08 2022] PUT /identity/v3/projects/1e9f02f94a10489abd3ab38e919d9b55/users/7836268aec65433bb23ff975acb8b5c3/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:08.192257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b438b-c583-476e-8dc8-7b837f8208b3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:08.205148 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b438b-c583-476e-8dc8-7b837f8208b3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:08.283003 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4d75f-e36e-466c-973a-5da009b78553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:08.283493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4d75f-e36e-466c-973a-5da009b78553 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:08.283723 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4d75f-e36e-466c-973a-5da009b78553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:08.283996 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4d75f-e36e-466c-973a-5da009b78553 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:08.284687 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4d75f-e36e-466c-973a-5da009b78553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:08.286310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4d75f-e36e-466c-973a-5da009b78553 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:08.296483 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1135/2271] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:34:08.300417 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c87bd4-9b46-4455-959d-371fa0631f93 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:08.311908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c87bd4-9b46-4455-959d-371fa0631f93 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:08.316865 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cae291-7fbc-4c92-99a1-44604fcc5354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:08.317572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cae291-7fbc-4c92-99a1-44604fcc5354 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:08.317857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cae291-7fbc-4c92-99a1-44604fcc5354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:08.318131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cae291-7fbc-4c92-99a1-44604fcc5354 admin admin] PATH_INFO: `/v3/projects/1e9f02f94a10489abd3ab38e919d9b55/users/7836268aec65433bb23ff975acb8b5c3/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:08.320876 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cae291-7fbc-4c92-99a1-44604fcc5354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e9f02f94a10489abd3ab38e919d9b55, user_id=7836268aec65433bb23ff975acb8b5c3, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:08.323603 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cae291-7fbc-4c92-99a1-44604fcc5354 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:08.334105 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1137/2272] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:34:08 2022] PUT /identity/v3/projects/1e9f02f94a10489abd3ab38e919d9b55/users/7836268aec65433bb23ff975acb8b5c3/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:08.345811 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b02bab-9f78-44f1-8427-09f10e3aab1b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:08.346100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b02bab-9f78-44f1-8427-09f10e3aab1b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:08.346372 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b02bab-9f78-44f1-8427-09f10e3aab1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:08.376823 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-53b02bab-9f78-44f1-8427-09f10e3aab1b None None] MFA Rules not processed for user `7836268aec65433bb23ff975acb8b5c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:08.378151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-53b02bab-9f78-44f1-8427-09f10e3aab1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:08.383054 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:08.471474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1136/2273] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:08 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:08.483136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cae291-7fbc-4c92-99a1-44604fcc5354 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:08.485063 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b8cae291-7fbc-4c92-99a1-44604fcc5354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:08.583096 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cae291-7fbc-4c92-99a1-44604fcc5354 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:08.588305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55917808-e79a-4317-8b85-bbf8aa0dd28d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:08.589007 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55917808-e79a-4317-8b85-bbf8aa0dd28d service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:08.589373 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55917808-e79a-4317-8b85-bbf8aa0dd28d service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:08.589745 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55917808-e79a-4317-8b85-bbf8aa0dd28d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:08.601148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55917808-e79a-4317-8b85-bbf8aa0dd28d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:08.603586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55917808-e79a-4317-8b85-bbf8aa0dd28d service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:08.613765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1138/2274] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:34:08 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:34:08.706493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b02bab-9f78-44f1-8427-09f10e3aab1b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:08.717771 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b02bab-9f78-44f1-8427-09f10e3aab1b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:08.721550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a7d92-b38b-46d0-b676-75892417ed14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:08.722186 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a7d92-b38b-46d0-b676-75892417ed14 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:08.722469 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a7d92-b38b-46d0-b676-75892417ed14 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:08.722740 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a7d92-b38b-46d0-b676-75892417ed14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:08.797435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a7d92-b38b-46d0-b676-75892417ed14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:08.798896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6a7d92-b38b-46d0-b676-75892417ed14 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:08.808653 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1137/2275] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:34:08 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:34:12.887590 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55917808-e79a-4317-8b85-bbf8aa0dd28d service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:12.949761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55917808-e79a-4317-8b85-bbf8aa0dd28d service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:12.954297 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfcbdc-8671-4a93-b221-e572a75bf02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:12.954940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfcbdc-8671-4a93-b221-e572a75bf02f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:12.955220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfcbdc-8671-4a93-b221-e572a75bf02f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:12.955495 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bfcbdc-8671-4a93-b221-e572a75bf02f admin admin] PATH_INFO: `/v3/users/61813b61fa1a4519b57dac3c8c758982` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:12.956650 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bfcbdc-8671-4a93-b221-e572a75bf02f admin admin] RBAC: Authorizing `identity:delete_user(user_id=61813b61fa1a4519b57dac3c8c758982)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:12.958621 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bfcbdc-8671-4a93-b221-e572a75bf02f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:13.006328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c1bfcbdc-8671-4a93-b221-e572a75bf02f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61813b61fa1a4519b57dac3c8c758982'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:13.014932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c1bfcbdc-8671-4a93-b221-e572a75bf02f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61813b61fa1a4519b57dac3c8c758982'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:13.023375 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c1bfcbdc-8671-4a93-b221-e572a75bf02f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61813b61fa1a4519b57dac3c8c758982'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:13.024035 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c1bfcbdc-8671-4a93-b221-e572a75bf02f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61813b61fa1a4519b57dac3c8c758982'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:13.025197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:13.038906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c1bfcbdc-8671-4a93-b221-e572a75bf02f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61813b61fa1a4519b57dac3c8c758982'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:13.044955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1139/2276] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:34:12 2022] DELETE /identity/v3/users/61813b61fa1a4519b57dac3c8c758982 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:13.049450 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a683e6-94dc-4dba-a8c3-402eb1ea5d2b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:13.049975 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a683e6-94dc-4dba-a8c3-402eb1ea5d2b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:13.050085 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a683e6-94dc-4dba-a8c3-402eb1ea5d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:13.069166 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f9a683e6-94dc-4dba-a8c3-402eb1ea5d2b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:13.070382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f9a683e6-94dc-4dba-a8c3-402eb1ea5d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:13.129861 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1138/2277] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:13.139815 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfcbdc-8671-4a93-b221-e572a75bf02f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:13.144676 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c1bfcbdc-8671-4a93-b221-e572a75bf02f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:13.215199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bfcbdc-8671-4a93-b221-e572a75bf02f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:13.219142 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f52b56-2fa0-4af9-a94f-5a386bc525ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:13.219769 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f52b56-2fa0-4af9-a94f-5a386bc525ef service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:13.220047 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f52b56-2fa0-4af9-a94f-5a386bc525ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:13.220321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f52b56-2fa0-4af9-a94f-5a386bc525ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:13.230996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f52b56-2fa0-4af9-a94f-5a386bc525ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:13.232599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f52b56-2fa0-4af9-a94f-5a386bc525ef service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:13.245562 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1140/2278] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:34:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:34:13.617963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a683e6-94dc-4dba-a8c3-402eb1ea5d2b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:13.619595 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f9a683e6-94dc-4dba-a8c3-402eb1ea5d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:13.640898 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a683e6-94dc-4dba-a8c3-402eb1ea5d2b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:13.651873 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd41de0-0413-4d78-94e7-5bb39a6b33f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:13.652001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd41de0-0413-4d78-94e7-5bb39a6b33f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:13.652252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd41de0-0413-4d78-94e7-5bb39a6b33f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:13.652537 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd41de0-0413-4d78-94e7-5bb39a6b33f5 admin admin] PATH_INFO: `/v3/projects/33e02f250f1f4531b44fa855799ee098` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:13.654590 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd41de0-0413-4d78-94e7-5bb39a6b33f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33e02f250f1f4531b44fa855799ee098)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:13.656542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd41de0-0413-4d78-94e7-5bb39a6b33f5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:13.748254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:13.763314 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73332a52-3a2c-49b8-a8fb-ae8a2573e335 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:13.763605 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73332a52-3a2c-49b8-a8fb-ae8a2573e335 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:13.763884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73332a52-3a2c-49b8-a8fb-ae8a2573e335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:13.767365 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fdd41de0-0413-4d78-94e7-5bb39a6b33f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33e02f250f1f4531b44fa855799ee098', 'request_id': 'req-fdd41de0-0413-4d78-94e7-5bb39a6b33f5'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:13.774439 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fdd41de0-0413-4d78-94e7-5bb39a6b33f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33e02f250f1f4531b44fa855799ee098', 'request_id': 'req-fdd41de0-0413-4d78-94e7-5bb39a6b33f5'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:13.788890 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fdd41de0-0413-4d78-94e7-5bb39a6b33f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33e02f250f1f4531b44fa855799ee098', 'request_id': 'req-fdd41de0-0413-4d78-94e7-5bb39a6b33f5'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:13.800883 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-73332a52-3a2c-49b8-a8fb-ae8a2573e335 None None] MFA Rules not processed for user `85626350279c40ea99da93b9ed6994b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:13.808339 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-73332a52-3a2c-49b8-a8fb-ae8a2573e335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:13.810505 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fdd41de0-0413-4d78-94e7-5bb39a6b33f5 admin admin] The token cache is being invalidate because project 33e02f250f1f4531b44fa855799ee098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:34:13.810940 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fdd41de0-0413-4d78-94e7-5bb39a6b33f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:13.811759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fdd41de0-0413-4d78-94e7-5bb39a6b33f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:13.814258 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1139/2279] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:34:13 2022] DELETE /identity/v3/projects/33e02f250f1f4531b44fa855799ee098 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:13.821067 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd41de0-0413-4d78-94e7-5bb39a6b33f5 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:13.824836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fdd41de0-0413-4d78-94e7-5bb39a6b33f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:13.923223 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1141/2280] 10.222.0.39 () {58 vars in 1009 bytes} [Thu Jul 28 22:34:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:13.945628 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd41de0-0413-4d78-94e7-5bb39a6b33f5 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:13.952298 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22292e8-f60e-4f25-a539-b7997306bff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:13.953050 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22292e8-f60e-4f25-a539-b7997306bff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:13.953334 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22292e8-f60e-4f25-a539-b7997306bff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:13.953665 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22292e8-f60e-4f25-a539-b7997306bff5 admin admin] PATH_INFO: `/v3/users/7836268aec65433bb23ff975acb8b5c3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:13.954872 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22292e8-f60e-4f25-a539-b7997306bff5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7836268aec65433bb23ff975acb8b5c3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:13.957132 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22292e8-f60e-4f25-a539-b7997306bff5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:13.997086 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73332a52-3a2c-49b8-a8fb-ae8a2573e335 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:13.998940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-73332a52-3a2c-49b8-a8fb-ae8a2573e335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:14.039790 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b22292e8-f60e-4f25-a539-b7997306bff5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7836268aec65433bb23ff975acb8b5c3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:14.049749 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b22292e8-f60e-4f25-a539-b7997306bff5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7836268aec65433bb23ff975acb8b5c3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:14.051431 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b22292e8-f60e-4f25-a539-b7997306bff5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7836268aec65433bb23ff975acb8b5c3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:14.052449 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b22292e8-f60e-4f25-a539-b7997306bff5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7836268aec65433bb23ff975acb8b5c3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:14.072195 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b22292e8-f60e-4f25-a539-b7997306bff5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7836268aec65433bb23ff975acb8b5c3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:14.092155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1140/2281] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:34:13 2022] DELETE /identity/v3/users/7836268aec65433bb23ff975acb8b5c3 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:34:14.103160 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73332a52-3a2c-49b8-a8fb-ae8a2573e335 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:14.160074 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:14.207613 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053dba5-434f-48e9-ab17-63e76dbec4a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9107872f147f6ab8698b67bb545af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:14.208540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053dba5-434f-48e9-ab17-63e76dbec4a0 service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:14.208874 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053dba5-434f-48e9-ab17-63e76dbec4a0 service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:14.210918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053dba5-434f-48e9-ab17-63e76dbec4a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:14.214494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3053dba5-434f-48e9-ab17-63e76dbec4a0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:14.285790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053dba5-434f-48e9-ab17-63e76dbec4a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:14.287434 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053dba5-434f-48e9-ab17-63e76dbec4a0 service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:14.297345 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1142/2282] 10.222.0.39 () {60 vars in 1384 bytes} [Thu Jul 28 22:34:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 22:34:14.636065 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22292e8-f60e-4f25-a539-b7997306bff5 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:14.637768 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b22292e8-f60e-4f25-a539-b7997306bff5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:14.728688 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:14.765897 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22292e8-f60e-4f25-a539-b7997306bff5 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:14.773207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2794e5a0-2016-485d-ae2e-b6e0c1e78991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:14.774010 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794e5a0-2016-485d-ae2e-b6e0c1e78991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:14.774297 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794e5a0-2016-485d-ae2e-b6e0c1e78991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:14.774572 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2794e5a0-2016-485d-ae2e-b6e0c1e78991 admin admin] PATH_INFO: `/v3/projects/1e9f02f94a10489abd3ab38e919d9b55` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:14.776431 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794e5a0-2016-485d-ae2e-b6e0c1e78991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e9f02f94a10489abd3ab38e919d9b55)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:14.778378 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2794e5a0-2016-485d-ae2e-b6e0c1e78991 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:14.848008 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2794e5a0-2016-485d-ae2e-b6e0c1e78991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e9f02f94a10489abd3ab38e919d9b55', 'request_id': 'req-2794e5a0-2016-485d-ae2e-b6e0c1e78991'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:14.856123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2794e5a0-2016-485d-ae2e-b6e0c1e78991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e9f02f94a10489abd3ab38e919d9b55', 'request_id': 'req-2794e5a0-2016-485d-ae2e-b6e0c1e78991'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:14.862740 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2794e5a0-2016-485d-ae2e-b6e0c1e78991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e9f02f94a10489abd3ab38e919d9b55', 'request_id': 'req-2794e5a0-2016-485d-ae2e-b6e0c1e78991'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:14.881089 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2794e5a0-2016-485d-ae2e-b6e0c1e78991 admin admin] The token cache is being invalidate because project 1e9f02f94a10489abd3ab38e919d9b55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:34:14.881555 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2794e5a0-2016-485d-ae2e-b6e0c1e78991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:14.882273 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2794e5a0-2016-485d-ae2e-b6e0c1e78991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:14.887566 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1141/2283] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:34:14 2022] DELETE /identity/v3/projects/1e9f02f94a10489abd3ab38e919d9b55 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:14.888804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605d741-3243-4053-8bb7-7edbb2b15948 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:14.889107 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605d741-3243-4053-8bb7-7edbb2b15948 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:14.889391 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605d741-3243-4053-8bb7-7edbb2b15948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:14.934628 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9605d741-3243-4053-8bb7-7edbb2b15948 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:14.935877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9605d741-3243-4053-8bb7-7edbb2b15948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:15.106378 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1143/2284] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:34:15.114658 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0208cbc-8dbc-412c-947c-0baddc860eb3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:15.115257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0208cbc-8dbc-412c-947c-0baddc860eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:15.115257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0208cbc-8dbc-412c-947c-0baddc860eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:15.154760 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e0208cbc-8dbc-412c-947c-0baddc860eb3 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:15.155988 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e0208cbc-8dbc-412c-947c-0baddc860eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:15.179499 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1142/2285] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:15.183972 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605d741-3243-4053-8bb7-7edbb2b15948 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:15.212218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605d741-3243-4053-8bb7-7edbb2b15948 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:15.221933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a254d-3bd3-4935-a4ea-5df1949ed77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:15.225289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a254d-3bd3-4935-a4ea-5df1949ed77d admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:15.225894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a254d-3bd3-4935-a4ea-5df1949ed77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:15.226409 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a254d-3bd3-4935-a4ea-5df1949ed77d admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:15.227567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a254d-3bd3-4935-a4ea-5df1949ed77d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:34:15.228056 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a254d-3bd3-4935-a4ea-5df1949ed77d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:15.232994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a254d-3bd3-4935-a4ea-5df1949ed77d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:15.250311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1144/2286] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:34:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:34:15.255973 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0208cbc-8dbc-412c-947c-0baddc860eb3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:15.276009 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0208cbc-8dbc-412c-947c-0baddc860eb3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:15.289756 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38901000-4c25-49d3-98c6-32a6ce134339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:15.290955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38901000-4c25-49d3-98c6-32a6ce134339 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:15.291399 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38901000-4c25-49d3-98c6-32a6ce134339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:15.291859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38901000-4c25-49d3-98c6-32a6ce134339 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:15.293641 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38901000-4c25-49d3-98c6-32a6ce134339 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:15.296334 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38901000-4c25-49d3-98c6-32a6ce134339 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:15.344113 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1143/2287] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:34:15 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:34:15.348145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a254d-3bd3-4935-a4ea-5df1949ed77d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:15.366614 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a254d-3bd3-4935-a4ea-5df1949ed77d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:15.459906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03184425-2391-4690-bc87-a6ee4080a039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:15.460548 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03184425-2391-4690-bc87-a6ee4080a039 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:15.460834 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03184425-2391-4690-bc87-a6ee4080a039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:15.461112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03184425-2391-4690-bc87-a6ee4080a039 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:15.461997 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03184425-2391-4690-bc87-a6ee4080a039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:15.463990 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03184425-2391-4690-bc87-a6ee4080a039 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:15.475229 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:15.496013 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1145/2288] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:34:15 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:34:15.500063 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38901000-4c25-49d3-98c6-32a6ce134339 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:15.514494 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38901000-4c25-49d3-98c6-32a6ce134339 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:15.519261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9911354-dce9-4d7f-a3b7-49059c9bab6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:15.520167 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9911354-dce9-4d7f-a3b7-49059c9bab6f admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:15.520859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9911354-dce9-4d7f-a3b7-49059c9bab6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:15.521217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9911354-dce9-4d7f-a3b7-49059c9bab6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:15.522764 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9911354-dce9-4d7f-a3b7-49059c9bab6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:15.525177 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9911354-dce9-4d7f-a3b7-49059c9bab6f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:15.539240 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1144/2289] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:34:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:34:15.543231 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03184425-2391-4690-bc87-a6ee4080a039 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:15.554085 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03184425-2391-4690-bc87-a6ee4080a039 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:15.562070 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074408d-48f0-4510-93ae-1322f74f55f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:15.562889 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074408d-48f0-4510-93ae-1322f74f55f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:15.563170 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074408d-48f0-4510-93ae-1322f74f55f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:15.563457 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074408d-48f0-4510-93ae-1322f74f55f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:15.564702 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074408d-48f0-4510-93ae-1322f74f55f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:15.566432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074408d-48f0-4510-93ae-1322f74f55f7 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:15.578161 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1146/2290] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:34:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:34:15.582373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9911354-dce9-4d7f-a3b7-49059c9bab6f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:15.594864 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9911354-dce9-4d7f-a3b7-49059c9bab6f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:15.600212 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a5ad0-38e8-45a2-a4af-1057b54da8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:15.600874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a5ad0-38e8-45a2-a4af-1057b54da8a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:15.601213 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a5ad0-38e8-45a2-a4af-1057b54da8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:15.603089 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a5ad0-38e8-45a2-a4af-1057b54da8a1 admin admin] PATH_INFO: `/v3/projects/a2710882473f4053b514e9548d8de725/users/731bae2ecd954266b2fafbc390d8a653/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:15.629446 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a5ad0-38e8-45a2-a4af-1057b54da8a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2710882473f4053b514e9548d8de725, user_id=731bae2ecd954266b2fafbc390d8a653, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:15.634136 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949a5ad0-38e8-45a2-a4af-1057b54da8a1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:15.667563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1145/2291] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:34:15 2022] PUT /identity/v3/projects/a2710882473f4053b514e9548d8de725/users/731bae2ecd954266b2fafbc390d8a653/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:15.669408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e88e09-269a-4bb7-b55d-bcb51257521a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:15.669723 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e88e09-269a-4bb7-b55d-bcb51257521a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:15.670065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e88e09-269a-4bb7-b55d-bcb51257521a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:15.713191 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-27e88e09-269a-4bb7-b55d-bcb51257521a None None] MFA Rules not processed for user `731bae2ecd954266b2fafbc390d8a653`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:15.714552 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-27e88e09-269a-4bb7-b55d-bcb51257521a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:15.841809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1147/2292] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:15.842484 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a5ad0-38e8-45a2-a4af-1057b54da8a1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:15.950728 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a5ad0-38e8-45a2-a4af-1057b54da8a1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:15.957827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67e5a8-c5a6-484c-8a86-403f7b923046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:15.959860 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67e5a8-c5a6-484c-8a86-403f7b923046 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:15.960255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67e5a8-c5a6-484c-8a86-403f7b923046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:15.960529 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67e5a8-c5a6-484c-8a86-403f7b923046 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:15.961221 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67e5a8-c5a6-484c-8a86-403f7b923046 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:15.963245 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67e5a8-c5a6-484c-8a86-403f7b923046 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:15.981377 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:15.999304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1146/2293] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:34:15 2022] POST /identity/v3/projects => generated 379 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:34:16.003088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e88e09-269a-4bb7-b55d-bcb51257521a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:16.016268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e88e09-269a-4bb7-b55d-bcb51257521a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:16.084174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3b0bb-59ae-4c55-94df-e8c2d9b9c22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:16.084830 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3b0bb-59ae-4c55-94df-e8c2d9b9c22b admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:16.085259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3b0bb-59ae-4c55-94df-e8c2d9b9c22b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:16.085645 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3b0bb-59ae-4c55-94df-e8c2d9b9c22b admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:16.086518 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3b0bb-59ae-4c55-94df-e8c2d9b9c22b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:16.088570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d3b0bb-59ae-4c55-94df-e8c2d9b9c22b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:16.116771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1148/2294] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:34:16 2022] POST /identity/v3/users => generated 290 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:34:16.118226 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67e5a8-c5a6-484c-8a86-403f7b923046 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:16.132540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67e5a8-c5a6-484c-8a86-403f7b923046 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:16.139185 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb760a-3b0b-4e5c-be46-5ab390731415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:16.141568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb760a-3b0b-4e5c-be46-5ab390731415 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:16.141568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb760a-3b0b-4e5c-be46-5ab390731415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:16.141568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fb760a-3b0b-4e5c-be46-5ab390731415 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:16.143029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb760a-3b0b-4e5c-be46-5ab390731415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:16.144718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fb760a-3b0b-4e5c-be46-5ab390731415 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:16.160161 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1147/2295] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:34:16.163289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3b0bb-59ae-4c55-94df-e8c2d9b9c22b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:16.175252 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3b0bb-59ae-4c55-94df-e8c2d9b9c22b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:16.180511 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2c778-1db7-49ff-8ebc-7459a8079d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:16.182901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2c778-1db7-49ff-8ebc-7459a8079d84 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:16.182901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2c778-1db7-49ff-8ebc-7459a8079d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:16.182901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2c778-1db7-49ff-8ebc-7459a8079d84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:16.182901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2c778-1db7-49ff-8ebc-7459a8079d84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:16.184478 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2c778-1db7-49ff-8ebc-7459a8079d84 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:16.196174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1149/2296] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:34:16.202634 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb760a-3b0b-4e5c-be46-5ab390731415 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:16.214400 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fb760a-3b0b-4e5c-be46-5ab390731415 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:16.222364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae319e0-7793-484f-afa2-82831b112260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:16.223059 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae319e0-7793-484f-afa2-82831b112260 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:16.223337 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae319e0-7793-484f-afa2-82831b112260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:16.223606 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae319e0-7793-484f-afa2-82831b112260 admin admin] PATH_INFO: `/v3/projects/1ddf0cea143d40cfb71b124047e15dfb/users/3f127e730e7a44478b74d61b3f549c23/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:16.244800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae319e0-7793-484f-afa2-82831b112260 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ddf0cea143d40cfb71b124047e15dfb, user_id=3f127e730e7a44478b74d61b3f549c23, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:16.247787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae319e0-7793-484f-afa2-82831b112260 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:16.263737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1148/2297] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:34:16 2022] PUT /identity/v3/projects/1ddf0cea143d40cfb71b124047e15dfb/users/3f127e730e7a44478b74d61b3f549c23/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:16.274597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b7c23-8178-47e2-a0af-f4f3eb72becc None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:16.275188 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b7c23-8178-47e2-a0af-f4f3eb72becc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:16.275817 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b7c23-8178-47e2-a0af-f4f3eb72becc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:16.311878 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3d6b7c23-8178-47e2-a0af-f4f3eb72becc None None] MFA Rules not processed for user `3f127e730e7a44478b74d61b3f549c23`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:16.313124 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3d6b7c23-8178-47e2-a0af-f4f3eb72becc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:16.367147 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:16.425070 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae319e0-7793-484f-afa2-82831b112260 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:16.425973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1150/2298] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:16.523714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae319e0-7793-484f-afa2-82831b112260 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:16.530228 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd5eb6-fe3b-4d85-af36-44818feee035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:16.531682 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd5eb6-fe3b-4d85-af36-44818feee035 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:16.531980 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd5eb6-fe3b-4d85-af36-44818feee035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:16.532253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fd5eb6-fe3b-4d85-af36-44818feee035 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:16.533093 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd5eb6-fe3b-4d85-af36-44818feee035 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:16.535924 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fd5eb6-fe3b-4d85-af36-44818feee035 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:16.567109 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1149/2299] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:34:16 2022] POST /identity/v3/projects => generated 377 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:34:16.571614 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b7c23-8178-47e2-a0af-f4f3eb72becc None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:16.585351 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b7c23-8178-47e2-a0af-f4f3eb72becc None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:16.667262 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33ed68-10cb-4133-a431-79018f5950a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:16.668151 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33ed68-10cb-4133-a431-79018f5950a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:16.668493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33ed68-10cb-4133-a431-79018f5950a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:16.668851 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a33ed68-10cb-4133-a431-79018f5950a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:16.669610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33ed68-10cb-4133-a431-79018f5950a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:16.671664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a33ed68-10cb-4133-a431-79018f5950a4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:16.699006 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1151/2300] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:34:16 2022] POST /identity/v3/users => generated 295 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:34:16.702280 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd5eb6-fe3b-4d85-af36-44818feee035 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:16.713452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fd5eb6-fe3b-4d85-af36-44818feee035 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:16.721472 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c60ec-b472-4309-a919-22b45468a8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:16.722122 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c60ec-b472-4309-a919-22b45468a8b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:16.722405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c60ec-b472-4309-a919-22b45468a8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:16.722677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80c60ec-b472-4309-a919-22b45468a8b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:16.723409 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c60ec-b472-4309-a919-22b45468a8b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:16.725952 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80c60ec-b472-4309-a919-22b45468a8b1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:16.738262 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1150/2301] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:34:16.742739 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33ed68-10cb-4133-a431-79018f5950a4 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:16.755641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a33ed68-10cb-4133-a431-79018f5950a4 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:16.760772 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1de3fb-3c0a-49b6-a553-46f1ae19cccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:16.761550 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1de3fb-3c0a-49b6-a553-46f1ae19cccd admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:16.761851 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1de3fb-3c0a-49b6-a553-46f1ae19cccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:16.762134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1de3fb-3c0a-49b6-a553-46f1ae19cccd admin admin] PATH_INFO: `/v3/projects/665e7763f41c478babd2068644aee621/users/3c78bcb7381e4481b59291d37ebeea2a/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:16.779903 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1de3fb-3c0a-49b6-a553-46f1ae19cccd admin admin] RBAC: Authorizing `identity:create_grant(project_id=665e7763f41c478babd2068644aee621, user_id=3c78bcb7381e4481b59291d37ebeea2a, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:16.782628 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1de3fb-3c0a-49b6-a553-46f1ae19cccd admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:16.805992 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1152/2302] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:34:16 2022] PUT /identity/v3/projects/665e7763f41c478babd2068644aee621/users/3c78bcb7381e4481b59291d37ebeea2a/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:34:16.807863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c60ec-b472-4309-a919-22b45468a8b1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:16.822265 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80c60ec-b472-4309-a919-22b45468a8b1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:16.919778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71687d-195f-4339-b98c-584a7f9b1a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:16.920747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71687d-195f-4339-b98c-584a7f9b1a93 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:16.921272 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71687d-195f-4339-b98c-584a7f9b1a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:16.922875 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71687d-195f-4339-b98c-584a7f9b1a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:16.923948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71687d-195f-4339-b98c-584a7f9b1a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:16.925963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71687d-195f-4339-b98c-584a7f9b1a93 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:16.939762 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1151/2303] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:34:16 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:34:16.943834 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1de3fb-3c0a-49b6-a553-46f1ae19cccd admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:16.963343 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1de3fb-3c0a-49b6-a553-46f1ae19cccd admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:16.969876 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc9889-5bdd-41ad-9449-a83e6b066700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:16.970716 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc9889-5bdd-41ad-9449-a83e6b066700 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:16.971094 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc9889-5bdd-41ad-9449-a83e6b066700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:16.971446 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc9889-5bdd-41ad-9449-a83e6b066700 admin admin] PATH_INFO: `/v3/projects/665e7763f41c478babd2068644aee621/users/3c78bcb7381e4481b59291d37ebeea2a/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:16.973708 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc9889-5bdd-41ad-9449-a83e6b066700 admin admin] RBAC: Authorizing `identity:create_grant(project_id=665e7763f41c478babd2068644aee621, user_id=3c78bcb7381e4481b59291d37ebeea2a, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:16.977465 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc9889-5bdd-41ad-9449-a83e6b066700 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:17.000542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1153/2304] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:34:16 2022] PUT /identity/v3/projects/665e7763f41c478babd2068644aee621/users/3c78bcb7381e4481b59291d37ebeea2a/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:17.002409 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea2034-4179-49b3-818d-5790f770d460 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:17.002712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea2034-4179-49b3-818d-5790f770d460 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:17.003052 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea2034-4179-49b3-818d-5790f770d460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:17.036407 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-09ea2034-4179-49b3-818d-5790f770d460 None None] MFA Rules not processed for user `3c78bcb7381e4481b59291d37ebeea2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:17.038525 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-09ea2034-4179-49b3-818d-5790f770d460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:17.057403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:17.139083 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1152/2305] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:17.147891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc9889-5bdd-41ad-9449-a83e6b066700 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:17.149510 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-12cc9889-5bdd-41ad-9449-a83e6b066700 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:17.229230 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc9889-5bdd-41ad-9449-a83e6b066700 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:17.235753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c75b36-af67-4190-995a-00c1dd405b7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:17.236714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c75b36-af67-4190-995a-00c1dd405b7b service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:17.237311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c75b36-af67-4190-995a-00c1dd405b7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:17.238058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c75b36-af67-4190-995a-00c1dd405b7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:17.249102 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c75b36-af67-4190-995a-00c1dd405b7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:17.252892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c75b36-af67-4190-995a-00c1dd405b7b service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:17.274906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1154/2306] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:34:17 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:34:17.332488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea2034-4179-49b3-818d-5790f770d460 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:17.334645 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-09ea2034-4179-49b3-818d-5790f770d460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:17.354335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea2034-4179-49b3-818d-5790f770d460 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:17.359440 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6a02e-f92e-4fe0-9279-6d7ac3db141c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:17.360120 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6a02e-f92e-4fe0-9279-6d7ac3db141c service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:17.360339 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6a02e-f92e-4fe0-9279-6d7ac3db141c service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:17.360670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6a02e-f92e-4fe0-9279-6d7ac3db141c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:17.439916 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6a02e-f92e-4fe0-9279-6d7ac3db141c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:17.441979 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6a02e-f92e-4fe0-9279-6d7ac3db141c service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:17.468832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1153/2307] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:34:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:34:19.980041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742ebc8-a60b-4707-ae1f-e15be1f9960f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:19.980590 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742ebc8-a60b-4707-ae1f-e15be1f9960f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:19.980590 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742ebc8-a60b-4707-ae1f-e15be1f9960f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:20.022955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b742ebc8-a60b-4707-ae1f-e15be1f9960f None None] MFA Rules not processed for user `d38b517910894f7fa4351a2120b466d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:20.024212 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b742ebc8-a60b-4707-ae1f-e15be1f9960f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:20.074078 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:20.145453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1155/2308] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:20.564801 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6a02e-f92e-4fe0-9279-6d7ac3db141c service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:20.566613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6fb6a02e-f92e-4fe0-9279-6d7ac3db141c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:20.689109 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6a02e-f92e-4fe0-9279-6d7ac3db141c service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:20.696267 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81550ab-9ff1-4ed1-92d6-88c5a7fe989e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:20.696909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81550ab-9ff1-4ed1-92d6-88c5a7fe989e service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:20.697374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81550ab-9ff1-4ed1-92d6-88c5a7fe989e service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:20.698341 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81550ab-9ff1-4ed1-92d6-88c5a7fe989e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:20.700167 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e81550ab-9ff1-4ed1-92d6-88c5a7fe989e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:20.712452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81550ab-9ff1-4ed1-92d6-88c5a7fe989e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:20.714132 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81550ab-9ff1-4ed1-92d6-88c5a7fe989e service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:20.723175 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1154/2309] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:34:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:34:21.191957 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742ebc8-a60b-4707-ae1f-e15be1f9960f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:21.317720 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742ebc8-a60b-4707-ae1f-e15be1f9960f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:21.323325 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443336e3-e3a6-4a81-a4a0-c509c2ad300a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:21.324044 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443336e3-e3a6-4a81-a4a0-c509c2ad300a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:21.324639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443336e3-e3a6-4a81-a4a0-c509c2ad300a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:21.324841 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443336e3-e3a6-4a81-a4a0-c509c2ad300a admin admin] PATH_INFO: `/v3/users/731bae2ecd954266b2fafbc390d8a653` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:21.326119 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443336e3-e3a6-4a81-a4a0-c509c2ad300a admin admin] RBAC: Authorizing `identity:delete_user(user_id=731bae2ecd954266b2fafbc390d8a653)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:21.328120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443336e3-e3a6-4a81-a4a0-c509c2ad300a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:21.404148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-443336e3-e3a6-4a81-a4a0-c509c2ad300a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '731bae2ecd954266b2fafbc390d8a653'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:21.417201 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-443336e3-e3a6-4a81-a4a0-c509c2ad300a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '731bae2ecd954266b2fafbc390d8a653'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:21.429504 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-443336e3-e3a6-4a81-a4a0-c509c2ad300a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '731bae2ecd954266b2fafbc390d8a653'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:21.430458 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-443336e3-e3a6-4a81-a4a0-c509c2ad300a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '731bae2ecd954266b2fafbc390d8a653'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:21.444467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-443336e3-e3a6-4a81-a4a0-c509c2ad300a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '731bae2ecd954266b2fafbc390d8a653'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:21.453836 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1156/2310] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:34:21 2022] DELETE /identity/v3/users/731bae2ecd954266b2fafbc390d8a653 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:34:21.458363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24b7b0-fd35-42f7-b7e7-2be41b97a975 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:21.458939 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24b7b0-fd35-42f7-b7e7-2be41b97a975 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:21.459467 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa24b7b0-fd35-42f7-b7e7-2be41b97a975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:21.495896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-fa24b7b0-fd35-42f7-b7e7-2be41b97a975 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:21.497408 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fa24b7b0-fd35-42f7-b7e7-2be41b97a975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:21.596867 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:21.627879 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1155/2311] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:34:21.636172 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443336e3-e3a6-4a81-a4a0-c509c2ad300a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:21.637679 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-443336e3-e3a6-4a81-a4a0-c509c2ad300a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:21.732682 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443336e3-e3a6-4a81-a4a0-c509c2ad300a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:21.737326 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f45fbc-c5fe-4f6d-ab17-5597386400e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:21.738095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f45fbc-c5fe-4f6d-ab17-5597386400e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:21.738385 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f45fbc-c5fe-4f6d-ab17-5597386400e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:21.738657 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f45fbc-c5fe-4f6d-ab17-5597386400e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:21.768650 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f45fbc-c5fe-4f6d-ab17-5597386400e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:21.770307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f45fbc-c5fe-4f6d-ab17-5597386400e5 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:21.776151 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:21.785283 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1157/2312] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:34:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:34:22.279736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24b7b0-fd35-42f7-b7e7-2be41b97a975 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:22.281590 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fa24b7b0-fd35-42f7-b7e7-2be41b97a975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:22.303598 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa24b7b0-fd35-42f7-b7e7-2be41b97a975 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:22.311818 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8f6e8-2b8f-42db-8d53-c6c86c81265d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:22.312817 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8f6e8-2b8f-42db-8d53-c6c86c81265d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:22.313256 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8f6e8-2b8f-42db-8d53-c6c86c81265d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:22.313724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8f6e8-2b8f-42db-8d53-c6c86c81265d admin admin] PATH_INFO: `/v3/projects/a2710882473f4053b514e9548d8de725` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:22.315192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8f6e8-2b8f-42db-8d53-c6c86c81265d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2710882473f4053b514e9548d8de725)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:22.321511 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8f6e8-2b8f-42db-8d53-c6c86c81265d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:22.434273 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-15d8f6e8-2b8f-42db-8d53-c6c86c81265d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2710882473f4053b514e9548d8de725', 'request_id': 'req-15d8f6e8-2b8f-42db-8d53-c6c86c81265d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:22.444189 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-15d8f6e8-2b8f-42db-8d53-c6c86c81265d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2710882473f4053b514e9548d8de725', 'request_id': 'req-15d8f6e8-2b8f-42db-8d53-c6c86c81265d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:22.452325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-15d8f6e8-2b8f-42db-8d53-c6c86c81265d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2710882473f4053b514e9548d8de725', 'request_id': 'req-15d8f6e8-2b8f-42db-8d53-c6c86c81265d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:22.491951 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-15d8f6e8-2b8f-42db-8d53-c6c86c81265d admin admin] The token cache is being invalidate because project a2710882473f4053b514e9548d8de725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:34:22.492303 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-15d8f6e8-2b8f-42db-8d53-c6c86c81265d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:22.495609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-15d8f6e8-2b8f-42db-8d53-c6c86c81265d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:22.497403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1156/2313] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:34:22 2022] DELETE /identity/v3/projects/a2710882473f4053b514e9548d8de725 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:22.502218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f45fbc-c5fe-4f6d-ab17-5597386400e5 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:22.527260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f45fbc-c5fe-4f6d-ab17-5597386400e5 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:22.634598 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ffa15-0c1b-4dfc-ac5a-a427e17b41e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:22.635321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ffa15-0c1b-4dfc-ac5a-a427e17b41e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:22.635612 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ffa15-0c1b-4dfc-ac5a-a427e17b41e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:22.636161 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48ffa15-0c1b-4dfc-ac5a-a427e17b41e4 admin admin] PATH_INFO: `/v3/users/3f127e730e7a44478b74d61b3f549c23` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:22.637513 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ffa15-0c1b-4dfc-ac5a-a427e17b41e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f127e730e7a44478b74d61b3f549c23)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:22.639491 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48ffa15-0c1b-4dfc-ac5a-a427e17b41e4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:22.852663 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b48ffa15-0c1b-4dfc-ac5a-a427e17b41e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f127e730e7a44478b74d61b3f549c23'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:22.872023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b48ffa15-0c1b-4dfc-ac5a-a427e17b41e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f127e730e7a44478b74d61b3f549c23'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:22.881376 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b48ffa15-0c1b-4dfc-ac5a-a427e17b41e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f127e730e7a44478b74d61b3f549c23'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:22.882266 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b48ffa15-0c1b-4dfc-ac5a-a427e17b41e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f127e730e7a44478b74d61b3f549c23'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:22.901485 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b48ffa15-0c1b-4dfc-ac5a-a427e17b41e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f127e730e7a44478b74d61b3f549c23'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:22.909483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1158/2314] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:34:22 2022] DELETE /identity/v3/users/3f127e730e7a44478b74d61b3f549c23 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:23.542277 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8f6e8-2b8f-42db-8d53-c6c86c81265d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:23.543957 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-15d8f6e8-2b8f-42db-8d53-c6c86c81265d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:23.566647 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8f6e8-2b8f-42db-8d53-c6c86c81265d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:23.663655 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe6a71-b7d9-4bbd-8a02-ff2262212c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:23.664395 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe6a71-b7d9-4bbd-8a02-ff2262212c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:23.664790 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe6a71-b7d9-4bbd-8a02-ff2262212c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:23.665163 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe6a71-b7d9-4bbd-8a02-ff2262212c31 admin admin] PATH_INFO: `/v3/projects/1ddf0cea143d40cfb71b124047e15dfb` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:23.666487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe6a71-b7d9-4bbd-8a02-ff2262212c31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ddf0cea143d40cfb71b124047e15dfb)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:23.668667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fe6a71-b7d9-4bbd-8a02-ff2262212c31 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:23.770431 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-80fe6a71-b7d9-4bbd-8a02-ff2262212c31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ddf0cea143d40cfb71b124047e15dfb', 'request_id': 'req-80fe6a71-b7d9-4bbd-8a02-ff2262212c31'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:23.795572 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-80fe6a71-b7d9-4bbd-8a02-ff2262212c31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ddf0cea143d40cfb71b124047e15dfb', 'request_id': 'req-80fe6a71-b7d9-4bbd-8a02-ff2262212c31'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:23.795572 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-80fe6a71-b7d9-4bbd-8a02-ff2262212c31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ddf0cea143d40cfb71b124047e15dfb', 'request_id': 'req-80fe6a71-b7d9-4bbd-8a02-ff2262212c31'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:23.812288 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-80fe6a71-b7d9-4bbd-8a02-ff2262212c31 admin admin] The token cache is being invalidate because project 1ddf0cea143d40cfb71b124047e15dfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:34:23.812541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-80fe6a71-b7d9-4bbd-8a02-ff2262212c31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:23.813369 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-80fe6a71-b7d9-4bbd-8a02-ff2262212c31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:23.815355 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1157/2315] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:34:23 2022] DELETE /identity/v3/projects/1ddf0cea143d40cfb71b124047e15dfb => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:23.820506 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ffa15-0c1b-4dfc-ac5a-a427e17b41e4 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:23.822048 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b48ffa15-0c1b-4dfc-ac5a-a427e17b41e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:23.835912 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48ffa15-0c1b-4dfc-ac5a-a427e17b41e4 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:23.914935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada5101-9083-4d2c-a0bd-6b9dc3a8227f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:23.915640 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada5101-9083-4d2c-a0bd-6b9dc3a8227f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:23.916036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada5101-9083-4d2c-a0bd-6b9dc3a8227f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:23.916430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada5101-9083-4d2c-a0bd-6b9dc3a8227f admin admin] PATH_INFO: `/v3/users/3c78bcb7381e4481b59291d37ebeea2a` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:23.917859 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada5101-9083-4d2c-a0bd-6b9dc3a8227f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c78bcb7381e4481b59291d37ebeea2a)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:23.920013 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada5101-9083-4d2c-a0bd-6b9dc3a8227f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:23.951251 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:24.003387 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8ada5101-9083-4d2c-a0bd-6b9dc3a8227f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c78bcb7381e4481b59291d37ebeea2a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:24.018186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8ada5101-9083-4d2c-a0bd-6b9dc3a8227f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c78bcb7381e4481b59291d37ebeea2a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:24.028439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8ada5101-9083-4d2c-a0bd-6b9dc3a8227f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c78bcb7381e4481b59291d37ebeea2a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:24.029299 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8ada5101-9083-4d2c-a0bd-6b9dc3a8227f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c78bcb7381e4481b59291d37ebeea2a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:24.049576 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-8ada5101-9083-4d2c-a0bd-6b9dc3a8227f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c78bcb7381e4481b59291d37ebeea2a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:24.072350 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1159/2316] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:34:23 2022] DELETE /identity/v3/users/3c78bcb7381e4481b59291d37ebeea2a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:34:24.636238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe6a71-b7d9-4bbd-8a02-ff2262212c31 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:24.648809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-80fe6a71-b7d9-4bbd-8a02-ff2262212c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:24.662290 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe6a71-b7d9-4bbd-8a02-ff2262212c31 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:24.670839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:24.743421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f919fbe-39d4-443a-b13f-94b5f5b377f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:24.744329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f919fbe-39d4-443a-b13f-94b5f5b377f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:24.744713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f919fbe-39d4-443a-b13f-94b5f5b377f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:24.745180 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f919fbe-39d4-443a-b13f-94b5f5b377f1 admin admin] PATH_INFO: `/v3/projects/665e7763f41c478babd2068644aee621` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:24.746616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f919fbe-39d4-443a-b13f-94b5f5b377f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=665e7763f41c478babd2068644aee621)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:24.748712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f919fbe-39d4-443a-b13f-94b5f5b377f1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:24.834476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7f919fbe-39d4-443a-b13f-94b5f5b377f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '665e7763f41c478babd2068644aee621', 'request_id': 'req-7f919fbe-39d4-443a-b13f-94b5f5b377f1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:24.841590 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7f919fbe-39d4-443a-b13f-94b5f5b377f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '665e7763f41c478babd2068644aee621', 'request_id': 'req-7f919fbe-39d4-443a-b13f-94b5f5b377f1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:24.853893 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7f919fbe-39d4-443a-b13f-94b5f5b377f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '665e7763f41c478babd2068644aee621', 'request_id': 'req-7f919fbe-39d4-443a-b13f-94b5f5b377f1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:24.869819 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7f919fbe-39d4-443a-b13f-94b5f5b377f1 admin admin] The token cache is being invalidate because project 665e7763f41c478babd2068644aee621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:34:24.870499 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7f919fbe-39d4-443a-b13f-94b5f5b377f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:24.871394 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7f919fbe-39d4-443a-b13f-94b5f5b377f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:24.873350 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1158/2317] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:34:24 2022] DELETE /identity/v3/projects/665e7763f41c478babd2068644aee621 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:24.879553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635e8e1-687d-4759-83cf-10edf9724c49 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:24.880454 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635e8e1-687d-4759-83cf-10edf9724c49 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:24.880897 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635e8e1-687d-4759-83cf-10edf9724c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:24.924676 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-4635e8e1-687d-4759-83cf-10edf9724c49 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:24.926165 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4635e8e1-687d-4759-83cf-10edf9724c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:25.008302 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1160/2318] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:25.017745 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02d481-d1aa-44b4-afc1-2409519a2885 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:25.018103 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02d481-d1aa-44b4-afc1-2409519a2885 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:25.018490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b02d481-d1aa-44b4-afc1-2409519a2885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:25.100913 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9b02d481-d1aa-44b4-afc1-2409519a2885 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:25.102667 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9b02d481-d1aa-44b4-afc1-2409519a2885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:25.120998 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1159/2319] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:25.122752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635e8e1-687d-4759-83cf-10edf9724c49 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:25.174470 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635e8e1-687d-4759-83cf-10edf9724c49 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:25.188541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f7422-dbbe-4346-bbae-1f8e6ef4690e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:25.189623 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f7422-dbbe-4346-bbae-1f8e6ef4690e admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:25.190157 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f7422-dbbe-4346-bbae-1f8e6ef4690e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:25.190778 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5f7422-dbbe-4346-bbae-1f8e6ef4690e admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:25.191869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f7422-dbbe-4346-bbae-1f8e6ef4690e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:34:25.192099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f7422-dbbe-4346-bbae-1f8e6ef4690e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:25.201658 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5f7422-dbbe-4346-bbae-1f8e6ef4690e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:25.238380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1161/2320] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:34:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:34:25.241021 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02d481-d1aa-44b4-afc1-2409519a2885 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:25.277216 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b02d481-d1aa-44b4-afc1-2409519a2885 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:25.283177 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc8439-e141-45d7-909f-d21b7861d27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:25.283827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc8439-e141-45d7-909f-d21b7861d27d admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:25.284111 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc8439-e141-45d7-909f-d21b7861d27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:25.284390 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc8439-e141-45d7-909f-d21b7861d27d admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:25.285056 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cc8439-e141-45d7-909f-d21b7861d27d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:25.287625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cc8439-e141-45d7-909f-d21b7861d27d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:25.323840 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1160/2321] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:34:25 2022] POST /identity/v3/projects => generated 387 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:34:25.327875 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f7422-dbbe-4346-bbae-1f8e6ef4690e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:25.351859 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5f7422-dbbe-4346-bbae-1f8e6ef4690e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:25.491937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d9161-a29f-4f3c-a245-42ac8b94a28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:25.492596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d9161-a29f-4f3c-a245-42ac8b94a28a admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:25.492877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d9161-a29f-4f3c-a245-42ac8b94a28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:25.493149 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d9161-a29f-4f3c-a245-42ac8b94a28a admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:25.493857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d9161-a29f-4f3c-a245-42ac8b94a28a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:25.496862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d9161-a29f-4f3c-a245-42ac8b94a28a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:25.538649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1162/2322] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:34:25 2022] POST /identity/v3/users => generated 294 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:34:25.539690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc8439-e141-45d7-909f-d21b7861d27d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:25.554948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc8439-e141-45d7-909f-d21b7861d27d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:25.560711 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37541f-ef77-4b12-9073-d3d49fdd4da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:25.561573 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37541f-ef77-4b12-9073-d3d49fdd4da8 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:25.562007 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37541f-ef77-4b12-9073-d3d49fdd4da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:25.562429 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c37541f-ef77-4b12-9073-d3d49fdd4da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:25.563782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37541f-ef77-4b12-9073-d3d49fdd4da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:25.565401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c37541f-ef77-4b12-9073-d3d49fdd4da8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:25.584300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1161/2323] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:34:25.585818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d9161-a29f-4f3c-a245-42ac8b94a28a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:25.597309 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d9161-a29f-4f3c-a245-42ac8b94a28a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:25.603107 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789d5865-3a9b-4d19-8d79-fce8da7dfe5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:25.603739 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d5865-3a9b-4d19-8d79-fce8da7dfe5d admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:25.604024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d5865-3a9b-4d19-8d79-fce8da7dfe5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:25.604300 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789d5865-3a9b-4d19-8d79-fce8da7dfe5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:25.605413 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d5865-3a9b-4d19-8d79-fce8da7dfe5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:25.609446 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789d5865-3a9b-4d19-8d79-fce8da7dfe5d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:25.621253 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1163/2324] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:34:25.622531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37541f-ef77-4b12-9073-d3d49fdd4da8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:25.635519 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c37541f-ef77-4b12-9073-d3d49fdd4da8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:25.640421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f22c8-136e-462c-93c9-b1b724d8c53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:25.641165 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f22c8-136e-462c-93c9-b1b724d8c53a admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:25.641513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f22c8-136e-462c-93c9-b1b724d8c53a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:25.641876 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f22c8-136e-462c-93c9-b1b724d8c53a admin admin] PATH_INFO: `/v3/projects/a3719832dee348638aeea50bcc2b5bbf/users/f93a4eb54d4e4c758590b8b39ad3501d/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:25.659143 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f22c8-136e-462c-93c9-b1b724d8c53a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3719832dee348638aeea50bcc2b5bbf, user_id=f93a4eb54d4e4c758590b8b39ad3501d, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:25.664400 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f22c8-136e-462c-93c9-b1b724d8c53a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:25.679948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1162/2325] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:34:25 2022] PUT /identity/v3/projects/a3719832dee348638aeea50bcc2b5bbf/users/f93a4eb54d4e4c758590b8b39ad3501d/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:25.692154 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3110e6-2581-4937-9da8-fcc838115189 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:25.692736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3110e6-2581-4937-9da8-fcc838115189 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:25.693333 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3110e6-2581-4937-9da8-fcc838115189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:25.702364 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:25.737678 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-2b3110e6-2581-4937-9da8-fcc838115189 None None] MFA Rules not processed for user `f93a4eb54d4e4c758590b8b39ad3501d`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:25.739415 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2b3110e6-2581-4937-9da8-fcc838115189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:25.825577 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1164/2326] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:25.833070 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f22c8-136e-462c-93c9-b1b724d8c53a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:25.896982 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:25.926216 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f22c8-136e-462c-93c9-b1b724d8c53a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:25.931040 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642e543-b7da-4c1e-bf02-28f0a8121718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:25.931664 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642e543-b7da-4c1e-bf02-28f0a8121718 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:25.931953 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642e543-b7da-4c1e-bf02-28f0a8121718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:25.932230 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642e543-b7da-4c1e-bf02-28f0a8121718 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:25.933017 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642e543-b7da-4c1e-bf02-28f0a8121718 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:25.934977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642e543-b7da-4c1e-bf02-28f0a8121718 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:25.972414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1163/2327] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:34:25 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:34:25.976424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3110e6-2581-4937-9da8-fcc838115189 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:25.989614 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3110e6-2581-4937-9da8-fcc838115189 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:26.054010 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a1ed6-1213-44f3-a7a7-6c4f077721c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:26.054637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a1ed6-1213-44f3-a7a7-6c4f077721c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:26.054924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a1ed6-1213-44f3-a7a7-6c4f077721c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:26.055204 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a1ed6-1213-44f3-a7a7-6c4f077721c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:26.055879 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a1ed6-1213-44f3-a7a7-6c4f077721c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:26.057821 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0a1ed6-1213-44f3-a7a7-6c4f077721c2 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:26.080851 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1165/2328] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:34:25 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:34:26.084853 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642e543-b7da-4c1e-bf02-28f0a8121718 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:26.096092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642e543-b7da-4c1e-bf02-28f0a8121718 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:26.114279 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0326d-d552-4fc5-a913-8f9ad0f27229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:26.115858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0326d-d552-4fc5-a913-8f9ad0f27229 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:26.116202 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0326d-d552-4fc5-a913-8f9ad0f27229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:26.116490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0326d-d552-4fc5-a913-8f9ad0f27229 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:26.117207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0326d-d552-4fc5-a913-8f9ad0f27229 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:26.118896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0326d-d552-4fc5-a913-8f9ad0f27229 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:26.135434 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a1ed6-1213-44f3-a7a7-6c4f077721c2 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:26.146588 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1164/2329] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:34:26.147178 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a1ed6-1213-44f3-a7a7-6c4f077721c2 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:26.158833 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaeef1-b2da-4a00-b435-184d7b00c54a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:26.162296 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaeef1-b2da-4a00-b435-184d7b00c54a admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:26.162677 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaeef1-b2da-4a00-b435-184d7b00c54a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:26.163570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaeef1-b2da-4a00-b435-184d7b00c54a admin admin] PATH_INFO: `/v3/projects/980ae3e8b1f94ddb8028db1cf169061d/users/43d1b1cd9906410d93b5a07f222941e3/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:26.189856 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feaeef1-b2da-4a00-b435-184d7b00c54a admin admin] RBAC: Authorizing `identity:create_grant(project_id=980ae3e8b1f94ddb8028db1cf169061d, user_id=43d1b1cd9906410d93b5a07f222941e3, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:26.192564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feaeef1-b2da-4a00-b435-184d7b00c54a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:26.207482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1166/2330] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:34:26 2022] PUT /identity/v3/projects/980ae3e8b1f94ddb8028db1cf169061d/users/43d1b1cd9906410d93b5a07f222941e3/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:26.212220 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0326d-d552-4fc5-a913-8f9ad0f27229 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:26.223791 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0326d-d552-4fc5-a913-8f9ad0f27229 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:26.295466 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c072aa6-33aa-4149-bf72-cd051f3cb07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:26.296449 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c072aa6-33aa-4149-bf72-cd051f3cb07c admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:26.296732 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c072aa6-33aa-4149-bf72-cd051f3cb07c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:26.297008 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c072aa6-33aa-4149-bf72-cd051f3cb07c admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:26.297736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c072aa6-33aa-4149-bf72-cd051f3cb07c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:26.299330 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c072aa6-33aa-4149-bf72-cd051f3cb07c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:26.310371 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1165/2331] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:34:26 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:34:26.314186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaeef1-b2da-4a00-b435-184d7b00c54a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:26.325479 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaeef1-b2da-4a00-b435-184d7b00c54a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:26.330123 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e5e43-9697-4bd7-bf65-072e25293fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:26.330778 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e5e43-9697-4bd7-bf65-072e25293fe4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:26.331059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e5e43-9697-4bd7-bf65-072e25293fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:26.331338 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8e5e43-9697-4bd7-bf65-072e25293fe4 admin admin] PATH_INFO: `/v3/projects/980ae3e8b1f94ddb8028db1cf169061d/users/43d1b1cd9906410d93b5a07f222941e3/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:26.333361 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e5e43-9697-4bd7-bf65-072e25293fe4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=980ae3e8b1f94ddb8028db1cf169061d, user_id=43d1b1cd9906410d93b5a07f222941e3, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:26.336424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8e5e43-9697-4bd7-bf65-072e25293fe4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:26.346322 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1167/2332] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:34:26 2022] PUT /identity/v3/projects/980ae3e8b1f94ddb8028db1cf169061d/users/43d1b1cd9906410d93b5a07f222941e3/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:26.357843 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cab48-2a76-409b-84da-e3ea8a3b6529 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:26.358124 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cab48-2a76-409b-84da-e3ea8a3b6529 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:26.358401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4cab48-2a76-409b-84da-e3ea8a3b6529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:26.387735 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4c4cab48-2a76-409b-84da-e3ea8a3b6529 None None] MFA Rules not processed for user `43d1b1cd9906410d93b5a07f222941e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:26.388935 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4c4cab48-2a76-409b-84da-e3ea8a3b6529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:26.467031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1166/2333] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:26.478454 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e5e43-9697-4bd7-bf65-072e25293fe4 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:26.480052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4b8e5e43-9697-4bd7-bf65-072e25293fe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:26.526989 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:26.539228 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8e5e43-9697-4bd7-bf65-072e25293fe4 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:26.542840 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d553e5b1-74fe-458b-8c15-c7b47aa160fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:26.543448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553e5b1-74fe-458b-8c15-c7b47aa160fc service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:26.543726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553e5b1-74fe-458b-8c15-c7b47aa160fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:26.543996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d553e5b1-74fe-458b-8c15-c7b47aa160fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:26.553221 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d553e5b1-74fe-458b-8c15-c7b47aa160fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:26.554727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d553e5b1-74fe-458b-8c15-c7b47aa160fc service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:26.563845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1168/2334] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:34:26 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:34:27.127329 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4cab48-2a76-409b-84da-e3ea8a3b6529 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:27.162105 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:27.192367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4cab48-2a76-409b-84da-e3ea8a3b6529 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:27.197462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb96f5-1ff0-41af-bad1-f5d914fca001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:27.198127 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb96f5-1ff0-41af-bad1-f5d914fca001 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:27.198411 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb96f5-1ff0-41af-bad1-f5d914fca001 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:27.198685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb96f5-1ff0-41af-bad1-f5d914fca001 admin admin] PATH_INFO: `/v3/users/f93a4eb54d4e4c758590b8b39ad3501d` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:27.199863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb96f5-1ff0-41af-bad1-f5d914fca001 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f93a4eb54d4e4c758590b8b39ad3501d)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:27.201830 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb96f5-1ff0-41af-bad1-f5d914fca001 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:27.249619 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-51eb96f5-1ff0-41af-bad1-f5d914fca001 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f93a4eb54d4e4c758590b8b39ad3501d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:27.257350 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-51eb96f5-1ff0-41af-bad1-f5d914fca001 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f93a4eb54d4e4c758590b8b39ad3501d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:27.258046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-51eb96f5-1ff0-41af-bad1-f5d914fca001 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f93a4eb54d4e4c758590b8b39ad3501d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:27.258664 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-51eb96f5-1ff0-41af-bad1-f5d914fca001 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f93a4eb54d4e4c758590b8b39ad3501d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:27.270373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-51eb96f5-1ff0-41af-bad1-f5d914fca001 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f93a4eb54d4e4c758590b8b39ad3501d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:27.284916 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1167/2335] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:34:27 2022] DELETE /identity/v3/users/f93a4eb54d4e4c758590b8b39ad3501d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:34:27.289174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f71d4-1c8b-4dc3-9402-521be3812819 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:27.289656 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f71d4-1c8b-4dc3-9402-521be3812819 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:27.289771 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f71d4-1c8b-4dc3-9402-521be3812819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:27.310753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e82f71d4-1c8b-4dc3-9402-521be3812819 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:27.311990 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e82f71d4-1c8b-4dc3-9402-521be3812819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:27.385811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1169/2336] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:27.401158 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb96f5-1ff0-41af-bad1-f5d914fca001 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:27.402816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-51eb96f5-1ff0-41af-bad1-f5d914fca001 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:27.476712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb96f5-1ff0-41af-bad1-f5d914fca001 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:27.480599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db55c1-8608-4dd6-9f1f-86b9a8de63aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:27.481167 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db55c1-8608-4dd6-9f1f-86b9a8de63aa service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:27.481453 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db55c1-8608-4dd6-9f1f-86b9a8de63aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:27.481757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db55c1-8608-4dd6-9f1f-86b9a8de63aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:27.491754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db55c1-8608-4dd6-9f1f-86b9a8de63aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:27.493192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db55c1-8608-4dd6-9f1f-86b9a8de63aa service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:27.502870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1168/2337] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:34:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:34:27.812391 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f71d4-1c8b-4dc3-9402-521be3812819 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:27.813998 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e82f71d4-1c8b-4dc3-9402-521be3812819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:27.840751 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f71d4-1c8b-4dc3-9402-521be3812819 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:27.854716 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd0609-358a-4158-8bd3-c851f7e1a584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:27.855409 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd0609-358a-4158-8bd3-c851f7e1a584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:27.855692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd0609-358a-4158-8bd3-c851f7e1a584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:27.857885 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd0609-358a-4158-8bd3-c851f7e1a584 admin admin] PATH_INFO: `/v3/projects/a3719832dee348638aeea50bcc2b5bbf` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:27.858035 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd0609-358a-4158-8bd3-c851f7e1a584 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3719832dee348638aeea50bcc2b5bbf)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:27.860671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecd0609-358a-4158-8bd3-c851f7e1a584 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:27.917810 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2ecd0609-358a-4158-8bd3-c851f7e1a584 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3719832dee348638aeea50bcc2b5bbf', 'request_id': 'req-2ecd0609-358a-4158-8bd3-c851f7e1a584'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:27.925561 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2ecd0609-358a-4158-8bd3-c851f7e1a584 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3719832dee348638aeea50bcc2b5bbf', 'request_id': 'req-2ecd0609-358a-4158-8bd3-c851f7e1a584'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:27.931849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2ecd0609-358a-4158-8bd3-c851f7e1a584 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3719832dee348638aeea50bcc2b5bbf', 'request_id': 'req-2ecd0609-358a-4158-8bd3-c851f7e1a584'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:27.943340 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2ecd0609-358a-4158-8bd3-c851f7e1a584 admin admin] The token cache is being invalidate because project a3719832dee348638aeea50bcc2b5bbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:34:27.943623 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2ecd0609-358a-4158-8bd3-c851f7e1a584 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:27.944271 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-2ecd0609-358a-4158-8bd3-c851f7e1a584 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:27.945800 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1170/2338] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:34:27 2022] DELETE /identity/v3/projects/a3719832dee348638aeea50bcc2b5bbf => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:27.949693 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db55c1-8608-4dd6-9f1f-86b9a8de63aa service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:27.960604 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db55c1-8608-4dd6-9f1f-86b9a8de63aa service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:28.029746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:28.043207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b38126-0fbc-4ea6-a95f-780c224b5b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:28.044921 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b38126-0fbc-4ea6-a95f-780c224b5b2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:28.045206 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b38126-0fbc-4ea6-a95f-780c224b5b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:28.045650 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b38126-0fbc-4ea6-a95f-780c224b5b2b admin admin] PATH_INFO: `/v3/users/43d1b1cd9906410d93b5a07f222941e3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:28.047791 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b38126-0fbc-4ea6-a95f-780c224b5b2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=43d1b1cd9906410d93b5a07f222941e3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:28.050370 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b38126-0fbc-4ea6-a95f-780c224b5b2b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:28.114586 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e7b38126-0fbc-4ea6-a95f-780c224b5b2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43d1b1cd9906410d93b5a07f222941e3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:28.129050 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e7b38126-0fbc-4ea6-a95f-780c224b5b2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43d1b1cd9906410d93b5a07f222941e3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:28.129850 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e7b38126-0fbc-4ea6-a95f-780c224b5b2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43d1b1cd9906410d93b5a07f222941e3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:28.130770 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e7b38126-0fbc-4ea6-a95f-780c224b5b2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43d1b1cd9906410d93b5a07f222941e3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:28.153658 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e7b38126-0fbc-4ea6-a95f-780c224b5b2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43d1b1cd9906410d93b5a07f222941e3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:28.172043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1169/2339] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:34:27 2022] DELETE /identity/v3/users/43d1b1cd9906410d93b5a07f222941e3 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:34:28.482048 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd0609-358a-4158-8bd3-c851f7e1a584 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:28.483611 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2ecd0609-358a-4158-8bd3-c851f7e1a584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:28.500250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd0609-358a-4158-8bd3-c851f7e1a584 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:28.572481 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298b0e33-887d-414f-a2f4-d34b1e50fb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:28.573250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b0e33-887d-414f-a2f4-d34b1e50fb55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:28.573627 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b0e33-887d-414f-a2f4-d34b1e50fb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:28.573973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298b0e33-887d-414f-a2f4-d34b1e50fb55 admin admin] PATH_INFO: `/v3/projects/980ae3e8b1f94ddb8028db1cf169061d` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:28.575207 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b0e33-887d-414f-a2f4-d34b1e50fb55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=980ae3e8b1f94ddb8028db1cf169061d)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:28.577200 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298b0e33-887d-414f-a2f4-d34b1e50fb55 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:28.590067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:28.633172 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-298b0e33-887d-414f-a2f4-d34b1e50fb55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '980ae3e8b1f94ddb8028db1cf169061d', 'request_id': 'req-298b0e33-887d-414f-a2f4-d34b1e50fb55'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:28.642981 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-298b0e33-887d-414f-a2f4-d34b1e50fb55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '980ae3e8b1f94ddb8028db1cf169061d', 'request_id': 'req-298b0e33-887d-414f-a2f4-d34b1e50fb55'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:28.650021 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-298b0e33-887d-414f-a2f4-d34b1e50fb55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '980ae3e8b1f94ddb8028db1cf169061d', 'request_id': 'req-298b0e33-887d-414f-a2f4-d34b1e50fb55'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:28.661132 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-298b0e33-887d-414f-a2f4-d34b1e50fb55 admin admin] The token cache is being invalidate because project 980ae3e8b1f94ddb8028db1cf169061d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:34:28.662050 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-298b0e33-887d-414f-a2f4-d34b1e50fb55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:28.662685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-298b0e33-887d-414f-a2f4-d34b1e50fb55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:28.664176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1171/2340] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:34:28 2022] DELETE /identity/v3/projects/980ae3e8b1f94ddb8028db1cf169061d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:28.668891 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20461f1-fa91-4566-a7f2-fcbc07ad6d4f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:28.669179 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20461f1-fa91-4566-a7f2-fcbc07ad6d4f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:28.669452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20461f1-fa91-4566-a7f2-fcbc07ad6d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:28.688413 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d20461f1-fa91-4566-a7f2-fcbc07ad6d4f None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:28.689639 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d20461f1-fa91-4566-a7f2-fcbc07ad6d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:28.749391 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1170/2341] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:28.813896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947f6ac-e70e-4d09-b4d1-5a1d65fe33c5 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:28.814112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947f6ac-e70e-4d09-b4d1-5a1d65fe33c5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:28.814356 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947f6ac-e70e-4d09-b4d1-5a1d65fe33c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:28.834395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e947f6ac-e70e-4d09-b4d1-5a1d65fe33c5 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:28.835599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e947f6ac-e70e-4d09-b4d1-5a1d65fe33c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:28.843391 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1172/2342] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:28.847521 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20461f1-fa91-4566-a7f2-fcbc07ad6d4f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:28.857710 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20461f1-fa91-4566-a7f2-fcbc07ad6d4f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:28.862383 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b565d0-ef70-4795-9532-ac1609de99f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:28.862992 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b565d0-ef70-4795-9532-ac1609de99f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:28.863274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b565d0-ef70-4795-9532-ac1609de99f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:28.863548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b565d0-ef70-4795-9532-ac1609de99f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:28.864252 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b565d0-ef70-4795-9532-ac1609de99f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:34:28.865178 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b565d0-ef70-4795-9532-ac1609de99f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:28.866829 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b565d0-ef70-4795-9532-ac1609de99f3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:28.878599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1171/2343] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:34:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:34:28.883379 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947f6ac-e70e-4d09-b4d1-5a1d65fe33c5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:28.892578 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947f6ac-e70e-4d09-b4d1-5a1d65fe33c5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:28.897871 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fef6c0-6713-4eab-8f04-825daa17a25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:28.898468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fef6c0-6713-4eab-8f04-825daa17a25f admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:28.898748 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fef6c0-6713-4eab-8f04-825daa17a25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:28.899019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fef6c0-6713-4eab-8f04-825daa17a25f admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:28.899688 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fef6c0-6713-4eab-8f04-825daa17a25f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:28.901603 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fef6c0-6713-4eab-8f04-825daa17a25f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:28.929408 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1173/2344] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:34:28 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:34:28.933458 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b565d0-ef70-4795-9532-ac1609de99f3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:28.943546 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b565d0-ef70-4795-9532-ac1609de99f3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:28.998123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b51785-740e-4c0c-a49c-1009bde1a9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:28.998777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b51785-740e-4c0c-a49c-1009bde1a9a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:28.999116 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b51785-740e-4c0c-a49c-1009bde1a9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:28.999426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b51785-740e-4c0c-a49c-1009bde1a9a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:29.000078 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b51785-740e-4c0c-a49c-1009bde1a9a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:29.002085 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b51785-740e-4c0c-a49c-1009bde1a9a2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:29.010788 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:29.025926 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1172/2345] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:34:28 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:34:29.030129 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fef6c0-6713-4eab-8f04-825daa17a25f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:29.040547 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fef6c0-6713-4eab-8f04-825daa17a25f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:29.045037 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86d304-8dd7-45dd-b673-c18e34992e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:29.045666 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86d304-8dd7-45dd-b673-c18e34992e7a admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:29.045965 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86d304-8dd7-45dd-b673-c18e34992e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:29.046216 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86d304-8dd7-45dd-b673-c18e34992e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:29.046904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86d304-8dd7-45dd-b673-c18e34992e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:29.048480 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86d304-8dd7-45dd-b673-c18e34992e7a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:29.058235 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1174/2346] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:34:29.061797 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b51785-740e-4c0c-a49c-1009bde1a9a2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:29.072865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b51785-740e-4c0c-a49c-1009bde1a9a2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:29.077456 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b189b-d6df-4d92-93e9-68e77af2c6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:29.078090 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b189b-d6df-4d92-93e9-68e77af2c6c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:29.078373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b189b-d6df-4d92-93e9-68e77af2c6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:29.078648 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b189b-d6df-4d92-93e9-68e77af2c6c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:29.079347 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b189b-d6df-4d92-93e9-68e77af2c6c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:29.080934 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486b189b-d6df-4d92-93e9-68e77af2c6c1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:29.092133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1173/2347] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:34:29.096195 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86d304-8dd7-45dd-b673-c18e34992e7a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:29.106266 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86d304-8dd7-45dd-b673-c18e34992e7a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:29.110737 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8e85b-18c6-40cb-8675-35f3b92c9b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:29.111384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8e85b-18c6-40cb-8675-35f3b92c9b21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:29.111664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8e85b-18c6-40cb-8675-35f3b92c9b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:29.111936 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8e85b-18c6-40cb-8675-35f3b92c9b21 admin admin] PATH_INFO: `/v3/projects/9780909925294bc6b29b069e6a6195f4/users/84e051e53e3a436b85648c92918194f0/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:29.126079 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8e85b-18c6-40cb-8675-35f3b92c9b21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9780909925294bc6b29b069e6a6195f4, user_id=84e051e53e3a436b85648c92918194f0, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:29.128946 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf8e85b-18c6-40cb-8675-35f3b92c9b21 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:29.142093 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1175/2348] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:34:29 2022] PUT /identity/v3/projects/9780909925294bc6b29b069e6a6195f4/users/84e051e53e3a436b85648c92918194f0/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:34:29.152870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791866e-8020-4f63-b701-c16bb7009da0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:29.153395 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791866e-8020-4f63-b701-c16bb7009da0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:29.153498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4791866e-8020-4f63-b701-c16bb7009da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:29.180525 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4791866e-8020-4f63-b701-c16bb7009da0 None None] MFA Rules not processed for user `84e051e53e3a436b85648c92918194f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:29.181763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4791866e-8020-4f63-b701-c16bb7009da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:29.244081 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1174/2349] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:29.255866 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8e85b-18c6-40cb-8675-35f3b92c9b21 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:29.257747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fdf8e85b-18c6-40cb-8675-35f3b92c9b21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:29.314284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8e85b-18c6-40cb-8675-35f3b92c9b21 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:29.318620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58be29-1061-497c-8580-43c18af3de5f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fecb440c0874ede9f3cf544cfed93ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:29.319241 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58be29-1061-497c-8580-43c18af3de5f service glance] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:29.319522 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58be29-1061-497c-8580-43c18af3de5f service glance] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:29.319796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58be29-1061-497c-8580-43c18af3de5f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:29.328674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58be29-1061-497c-8580-43c18af3de5f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:29.330337 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c58be29-1061-497c-8580-43c18af3de5f service glance] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:29.340089 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1176/2350] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:34:29 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:34:33.796040 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885f727-88aa-41a8-91b7-8fb0fc0651c9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:33.796529 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885f727-88aa-41a8-91b7-8fb0fc0651c9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:33.796610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a885f727-88aa-41a8-91b7-8fb0fc0651c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:33.824140 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a885f727-88aa-41a8-91b7-8fb0fc0651c9 None None] MFA Rules not processed for user `85626350279c40ea99da93b9ed6994b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:33.825373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a885f727-88aa-41a8-91b7-8fb0fc0651c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:33.887717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1175/2351] 10.222.0.39 () {58 vars in 1009 bytes} [Thu Jul 28 22:34:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:33.906880 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58be29-1061-497c-8580-43c18af3de5f service glance] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:33.908315 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8c58be29-1061-497c-8580-43c18af3de5f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:33.956521 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:33.987081 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58be29-1061-497c-8580-43c18af3de5f service glance] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:33.992827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f531eb4-5d74-4a19-bfc5-2ee431a620cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9107872f147f6ab8698b67bb545af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:33.993451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f531eb4-5d74-4a19-bfc5-2ee431a620cb service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:33.993765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f531eb4-5d74-4a19-bfc5-2ee431a620cb service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:33.994043 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f531eb4-5d74-4a19-bfc5-2ee431a620cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:34.002611 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f531eb4-5d74-4a19-bfc5-2ee431a620cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:34.004454 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f531eb4-5d74-4a19-bfc5-2ee431a620cb service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:34.013419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1177/2352] 10.222.0.39 () {60 vars in 1384 bytes} [Thu Jul 28 22:34:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 22:34:43.878891 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:INSERT=3,keystone:SELECT=64 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:44.010134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=14 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:48.305394 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885f727-88aa-41a8-91b7-8fb0fc0651c9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:48.374055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a885f727-88aa-41a8-91b7-8fb0fc0651c9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:48.378836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2663147-4991-47dd-8bcd-ec7bb0d5f0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:48.379473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2663147-4991-47dd-8bcd-ec7bb0d5f0bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:48.379754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2663147-4991-47dd-8bcd-ec7bb0d5f0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:48.380030 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2663147-4991-47dd-8bcd-ec7bb0d5f0bd admin admin] PATH_INFO: `/v3/users/84e051e53e3a436b85648c92918194f0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:48.381881 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2663147-4991-47dd-8bcd-ec7bb0d5f0bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=84e051e53e3a436b85648c92918194f0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:48.383803 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2663147-4991-47dd-8bcd-ec7bb0d5f0bd admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:48.429273 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f2663147-4991-47dd-8bcd-ec7bb0d5f0bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84e051e53e3a436b85648c92918194f0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:48.436156 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f2663147-4991-47dd-8bcd-ec7bb0d5f0bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84e051e53e3a436b85648c92918194f0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:48.436842 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f2663147-4991-47dd-8bcd-ec7bb0d5f0bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84e051e53e3a436b85648c92918194f0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:48.437498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f2663147-4991-47dd-8bcd-ec7bb0d5f0bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84e051e53e3a436b85648c92918194f0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:48.450656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f2663147-4991-47dd-8bcd-ec7bb0d5f0bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84e051e53e3a436b85648c92918194f0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:48.463118 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1176/2353] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:34:48 2022] DELETE /identity/v3/users/84e051e53e3a436b85648c92918194f0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:48.467781 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a511830-de81-46b9-b5c1-f5c7c131c117 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:48.468258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a511830-de81-46b9-b5c1-f5c7c131c117 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:48.468365 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a511830-de81-46b9-b5c1-f5c7c131c117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:48.488939 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-9a511830-de81-46b9-b5c1-f5c7c131c117 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:48.490980 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9a511830-de81-46b9-b5c1-f5c7c131c117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:48.547238 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1178/2354] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:48.557496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2663147-4991-47dd-8bcd-ec7bb0d5f0bd admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:48.558935 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f2663147-4991-47dd-8bcd-ec7bb0d5f0bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:48.616761 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2663147-4991-47dd-8bcd-ec7bb0d5f0bd admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:48.620557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf06011-b67f-4a39-a954-d2b59cd02e2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:48.621174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf06011-b67f-4a39-a954-d2b59cd02e2a service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:48.621459 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf06011-b67f-4a39-a954-d2b59cd02e2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:48.621761 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf06011-b67f-4a39-a954-d2b59cd02e2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:48.632144 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf06011-b67f-4a39-a954-d2b59cd02e2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:48.633606 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf06011-b67f-4a39-a954-d2b59cd02e2a service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:48.646781 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1177/2355] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:34:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:34:48.924566 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a511830-de81-46b9-b5c1-f5c7c131c117 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:48.926196 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9a511830-de81-46b9-b5c1-f5c7c131c117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:48.938381 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a511830-de81-46b9-b5c1-f5c7c131c117 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:48.944117 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6cc1ef-407f-4ba9-a251-e1f0f58cadfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:48.944824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cc1ef-407f-4ba9-a251-e1f0f58cadfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:48.945155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cc1ef-407f-4ba9-a251-e1f0f58cadfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:48.945609 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6cc1ef-407f-4ba9-a251-e1f0f58cadfd admin admin] PATH_INFO: `/v3/projects/9780909925294bc6b29b069e6a6195f4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:48.946754 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6cc1ef-407f-4ba9-a251-e1f0f58cadfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9780909925294bc6b29b069e6a6195f4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:48.948579 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6cc1ef-407f-4ba9-a251-e1f0f58cadfd admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:49.004854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ae6cc1ef-407f-4ba9-a251-e1f0f58cadfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9780909925294bc6b29b069e6a6195f4', 'request_id': 'req-ae6cc1ef-407f-4ba9-a251-e1f0f58cadfd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:49.012880 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ae6cc1ef-407f-4ba9-a251-e1f0f58cadfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9780909925294bc6b29b069e6a6195f4', 'request_id': 'req-ae6cc1ef-407f-4ba9-a251-e1f0f58cadfd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:49.018596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ae6cc1ef-407f-4ba9-a251-e1f0f58cadfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9780909925294bc6b29b069e6a6195f4', 'request_id': 'req-ae6cc1ef-407f-4ba9-a251-e1f0f58cadfd'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:49.030467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ae6cc1ef-407f-4ba9-a251-e1f0f58cadfd admin admin] The token cache is being invalidate because project 9780909925294bc6b29b069e6a6195f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:34:49.030754 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ae6cc1ef-407f-4ba9-a251-e1f0f58cadfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:49.031381 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-ae6cc1ef-407f-4ba9-a251-e1f0f58cadfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:34:49.032944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1179/2356] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:34:48 2022] DELETE /identity/v3/projects/9780909925294bc6b29b069e6a6195f4 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:49.038483 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d301898-5286-44cc-ab5f-75312e7ab9d9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:49.038772 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d301898-5286-44cc-ab5f-75312e7ab9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:49.039049 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d301898-5286-44cc-ab5f-75312e7ab9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:49.058810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2d301898-5286-44cc-ab5f-75312e7ab9d9 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:49.060517 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2d301898-5286-44cc-ab5f-75312e7ab9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:49.077753 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:49.122094 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1178/2357] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:49.131814 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7c3b8-a16f-4958-99ac-2b01b7b94c4c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:49.132097 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7c3b8-a16f-4958-99ac-2b01b7b94c4c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:49.132363 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec7c3b8-a16f-4958-99ac-2b01b7b94c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:49.152341 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8ec7c3b8-a16f-4958-99ac-2b01b7b94c4c None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:49.153552 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8ec7c3b8-a16f-4958-99ac-2b01b7b94c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:49.161344 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1180/2358] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:49.166110 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d301898-5286-44cc-ab5f-75312e7ab9d9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:49.175959 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d301898-5286-44cc-ab5f-75312e7ab9d9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:49.181146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bfa08-3521-440a-9dee-16dc6d4c4ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:49.181775 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bfa08-3521-440a-9dee-16dc6d4c4ebf admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:49.182053 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bfa08-3521-440a-9dee-16dc6d4c4ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:49.182324 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bfa08-3521-440a-9dee-16dc6d4c4ebf admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:49.183018 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677bfa08-3521-440a-9dee-16dc6d4c4ebf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:34:49.183319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677bfa08-3521-440a-9dee-16dc6d4c4ebf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:49.184928 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677bfa08-3521-440a-9dee-16dc6d4c4ebf admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:49.196374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1179/2359] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:34:49.200421 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7c3b8-a16f-4958-99ac-2b01b7b94c4c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:49.210227 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec7c3b8-a16f-4958-99ac-2b01b7b94c4c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:49.214870 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e92be-5187-4735-b9dc-6eb212c9c02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:49.215441 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e92be-5187-4735-b9dc-6eb212c9c02b admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:49.215720 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e92be-5187-4735-b9dc-6eb212c9c02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:49.215992 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e92be-5187-4735-b9dc-6eb212c9c02b admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:49.216654 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e92be-5187-4735-b9dc-6eb212c9c02b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:49.218616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e92be-5187-4735-b9dc-6eb212c9c02b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:49.247228 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1181/2360] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:34:49 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:34:49.251446 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bfa08-3521-440a-9dee-16dc6d4c4ebf admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:49.262693 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bfa08-3521-440a-9dee-16dc6d4c4ebf admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:49.319367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b12a17-9019-4705-b819-23bf5ef8a841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:49.320022 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b12a17-9019-4705-b819-23bf5ef8a841 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:49.320186 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b12a17-9019-4705-b819-23bf5ef8a841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:49.320467 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b12a17-9019-4705-b819-23bf5ef8a841 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:49.321139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b12a17-9019-4705-b819-23bf5ef8a841 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:49.323082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b12a17-9019-4705-b819-23bf5ef8a841 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:49.345317 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1180/2361] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:34:49 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:34:49.349711 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e92be-5187-4735-b9dc-6eb212c9c02b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:49.359818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e92be-5187-4735-b9dc-6eb212c9c02b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:49.364364 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718a679-fa42-4371-b1aa-44179fc51786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:49.365172 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718a679-fa42-4371-b1aa-44179fc51786 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:49.365472 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718a679-fa42-4371-b1aa-44179fc51786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:49.365777 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718a679-fa42-4371-b1aa-44179fc51786 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:49.366470 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718a679-fa42-4371-b1aa-44179fc51786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:49.368063 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718a679-fa42-4371-b1aa-44179fc51786 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:49.378091 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1182/2362] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:34:49.382470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b12a17-9019-4705-b819-23bf5ef8a841 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:49.392379 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b12a17-9019-4705-b819-23bf5ef8a841 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:49.397905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b475ac6-9b50-4832-8476-d2f9a2693560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:49.398531 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b475ac6-9b50-4832-8476-d2f9a2693560 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:49.398811 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b475ac6-9b50-4832-8476-d2f9a2693560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:49.399083 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b475ac6-9b50-4832-8476-d2f9a2693560 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:49.399777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b475ac6-9b50-4832-8476-d2f9a2693560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:49.401356 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b475ac6-9b50-4832-8476-d2f9a2693560 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:49.411723 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1181/2363] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:34:49.415912 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718a679-fa42-4371-b1aa-44179fc51786 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:49.425641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718a679-fa42-4371-b1aa-44179fc51786 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:49.430471 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df24184-d608-4d35-b160-4e66cb6b2d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:49.431126 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df24184-d608-4d35-b160-4e66cb6b2d87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:49.431406 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df24184-d608-4d35-b160-4e66cb6b2d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:49.431684 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df24184-d608-4d35-b160-4e66cb6b2d87 admin admin] PATH_INFO: `/v3/projects/f5144f6ac53848fe84bdcb15e5242f1d/users/d649218b6ae047c480de9640727aaded/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:49.447399 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df24184-d608-4d35-b160-4e66cb6b2d87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5144f6ac53848fe84bdcb15e5242f1d, user_id=d649218b6ae047c480de9640727aaded, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:49.450164 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df24184-d608-4d35-b160-4e66cb6b2d87 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:49.463380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1183/2364] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:34:49 2022] PUT /identity/v3/projects/f5144f6ac53848fe84bdcb15e5242f1d/users/d649218b6ae047c480de9640727aaded/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:34:49.468490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520810c-d60a-4d6e-881c-4400c8bb7dd5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:49.468786 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520810c-d60a-4d6e-881c-4400c8bb7dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:49.469067 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520810c-d60a-4d6e-881c-4400c8bb7dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:49.487776 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4520810c-d60a-4d6e-881c-4400c8bb7dd5 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:49.488992 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4520810c-d60a-4d6e-881c-4400c8bb7dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:49.547456 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1182/2365] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:49.557600 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df24184-d608-4d35-b160-4e66cb6b2d87 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:49.559352 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1df24184-d608-4d35-b160-4e66cb6b2d87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:49.581519 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:49.620157 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df24184-d608-4d35-b160-4e66cb6b2d87 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:49.623743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11648580-07c2-480d-9e58-3cdfccf8d2a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:49.624353 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11648580-07c2-480d-9e58-3cdfccf8d2a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:49.624635 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11648580-07c2-480d-9e58-3cdfccf8d2a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:49.624908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11648580-07c2-480d-9e58-3cdfccf8d2a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:49.635212 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11648580-07c2-480d-9e58-3cdfccf8d2a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:49.636632 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11648580-07c2-480d-9e58-3cdfccf8d2a7 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:49.645918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1184/2366] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:34:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:34:50.415312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c3d75-8a95-4851-80c2-2c2058b495e9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:50.415824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c3d75-8a95-4851-80c2-2c2058b495e9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:50.415922 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1c3d75-8a95-4851-80c2-2c2058b495e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:50.444895 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5a1c3d75-8a95-4851-80c2-2c2058b495e9 None None] MFA Rules not processed for user `d649218b6ae047c480de9640727aaded`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:50.446148 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5a1c3d75-8a95-4851-80c2-2c2058b495e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:50.464608 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:34:50.517519 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1183/2367] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:50.529181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11648580-07c2-480d-9e58-3cdfccf8d2a7 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:50.530749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-11648580-07c2-480d-9e58-3cdfccf8d2a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:50.589054 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11648580-07c2-480d-9e58-3cdfccf8d2a7 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:50.593105 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d0f0d-70ec-4cd4-889a-ade5f8f64128 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:50.593852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d0f0d-70ec-4cd4-889a-ade5f8f64128 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:50.594268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d0f0d-70ec-4cd4-889a-ade5f8f64128 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:50.594617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d0f0d-70ec-4cd4-889a-ade5f8f64128 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:50.603727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d0f0d-70ec-4cd4-889a-ade5f8f64128 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:50.605148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d0f0d-70ec-4cd4-889a-ade5f8f64128 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:50.614679 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1185/2368] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:34:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:34:51.966055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9186309-7b33-487a-abe9-194bd8a89a7b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:51.966538 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9186309-7b33-487a-abe9-194bd8a89a7b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:51.966658 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9186309-7b33-487a-abe9-194bd8a89a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:51.986839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d9186309-7b33-487a-abe9-194bd8a89a7b None None] MFA Rules not processed for user `d649218b6ae047c480de9640727aaded`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:51.988052 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d9186309-7b33-487a-abe9-194bd8a89a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:51.996189 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1184/2369] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:34:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:52.035532 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ca979-d7c2-450a-abe7-2f9cb0793310 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:52.035970 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ca979-d7c2-450a-abe7-2f9cb0793310 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:52.036085 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ca979-d7c2-450a-abe7-2f9cb0793310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:52.057541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-391ca979-d7c2-450a-abe7-2f9cb0793310 None None] MFA Rules not processed for user `85626350279c40ea99da93b9ed6994b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:34:52.058752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-391ca979-d7c2-450a-abe7-2f9cb0793310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:52.116173 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1186/2370] 10.222.0.39 () {58 vars in 967 bytes} [Thu Jul 28 22:34:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:34:52.128759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9186309-7b33-487a-abe9-194bd8a89a7b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:34:52.130233 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d9186309-7b33-487a-abe9-194bd8a89a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:34:52.233437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9186309-7b33-487a-abe9-194bd8a89a7b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:34:52.239423 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4825e-a6c6-4ed4-a472-a14ec7f1e002 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9107872f147f6ab8698b67bb545af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:34:52.241130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4825e-a6c6-4ed4-a472-a14ec7f1e002 service swift] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:34:52.242902 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4825e-a6c6-4ed4-a472-a14ec7f1e002 service swift] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:34:52.242902 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4825e-a6c6-4ed4-a472-a14ec7f1e002 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:34:52.251619 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4825e-a6c6-4ed4-a472-a14ec7f1e002 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:34:52.253726 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e4825e-a6c6-4ed4-a472-a14ec7f1e002 service swift] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:34:52.263064 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1185/2371] 10.222.0.39 () {60 vars in 1384 bytes} [Thu Jul 28 22:34:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 22:35:02.108863 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=68 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:02.259796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=54 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:08.507057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce151b-658d-44c6-9e06-ef1d4a4f3610 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:08.507536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce151b-658d-44c6-9e06-ef1d4a4f3610 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:08.507633 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce151b-658d-44c6-9e06-ef1d4a4f3610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:08.526174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-39ce151b-658d-44c6-9e06-ef1d4a4f3610 None None] MFA Rules not processed for user `d649218b6ae047c480de9640727aaded`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:08.527382 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-39ce151b-658d-44c6-9e06-ef1d4a4f3610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:08.534190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1187/2372] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:08.840582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4825e-a6c6-4ed4-a472-a14ec7f1e002 service swift] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:08.850781 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4825e-a6c6-4ed4-a472-a14ec7f1e002 service swift] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:08.854684 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613239b-1001-467d-8518-6717903a5b7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:08.855372 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613239b-1001-467d-8518-6717903a5b7f service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:08.855658 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613239b-1001-467d-8518-6717903a5b7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:08.855935 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d613239b-1001-467d-8518-6717903a5b7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:08.857732 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d613239b-1001-467d-8518-6717903a5b7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:08.871172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613239b-1001-467d-8518-6717903a5b7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:08.872614 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d613239b-1001-467d-8518-6717903a5b7f service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:08.883609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1186/2373] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:35:12.405327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce151b-658d-44c6-9e06-ef1d4a4f3610 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:12.407836 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-39ce151b-658d-44c6-9e06-ef1d4a4f3610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:12.497912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613239b-1001-467d-8518-6717903a5b7f service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:12.499466 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d613239b-1001-467d-8518-6717903a5b7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:12.519422 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce151b-658d-44c6-9e06-ef1d4a4f3610 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:12.525850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849067d5-685f-48ce-8696-6b2ac84ee77b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe003cd7bfbe4b92b1d64d300c3e7d8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:12.526495 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849067d5-685f-48ce-8696-6b2ac84ee77b service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:12.526640 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d613239b-1001-467d-8518-6717903a5b7f service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:12.526815 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849067d5-685f-48ce-8696-6b2ac84ee77b service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:12.527045 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849067d5-685f-48ce-8696-6b2ac84ee77b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:12.528942 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-849067d5-685f-48ce-8696-6b2ac84ee77b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:12.531924 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c82c7-62b9-4618-96e0-1deb8276c285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:12.532594 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c82c7-62b9-4618-96e0-1deb8276c285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:12.532882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c82c7-62b9-4618-96e0-1deb8276c285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:12.533216 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c82c7-62b9-4618-96e0-1deb8276c285 admin admin] PATH_INFO: `/v3/users/7476eccb0a6f4195936bec75f7f80c7d` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:12.534520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c82c7-62b9-4618-96e0-1deb8276c285 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7476eccb0a6f4195936bec75f7f80c7d)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:12.536515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c82c7-62b9-4618-96e0-1deb8276c285 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:12.542316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849067d5-685f-48ce-8696-6b2ac84ee77b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:12.544003 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849067d5-685f-48ce-8696-6b2ac84ee77b service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:12.555642 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1188/2374] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:35:12.608170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a05c82c7-62b9-4618-96e0-1deb8276c285 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7476eccb0a6f4195936bec75f7f80c7d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:12.615494 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a05c82c7-62b9-4618-96e0-1deb8276c285 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7476eccb0a6f4195936bec75f7f80c7d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:12.616299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a05c82c7-62b9-4618-96e0-1deb8276c285 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7476eccb0a6f4195936bec75f7f80c7d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:12.617010 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a05c82c7-62b9-4618-96e0-1deb8276c285 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7476eccb0a6f4195936bec75f7f80c7d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:12.632135 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a05c82c7-62b9-4618-96e0-1deb8276c285 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7476eccb0a6f4195936bec75f7f80c7d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:12.653155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1187/2375] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:35:12 2022] DELETE /identity/v3/users/7476eccb0a6f4195936bec75f7f80c7d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:12.838562 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849067d5-685f-48ce-8696-6b2ac84ee77b service placement] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:12.850972 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849067d5-685f-48ce-8696-6b2ac84ee77b service placement] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:12.930235 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a8e5d-3e09-45fe-8683-c3b54d95448d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:12.930984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a8e5d-3e09-45fe-8683-c3b54d95448d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:12.931341 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a8e5d-3e09-45fe-8683-c3b54d95448d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:12.931756 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a8e5d-3e09-45fe-8683-c3b54d95448d admin admin] PATH_INFO: `/v3/projects/d829cbd8678145668ea811e01093fcfc` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:12.933098 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a8e5d-3e09-45fe-8683-c3b54d95448d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d829cbd8678145668ea811e01093fcfc)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:12.935074 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a8e5d-3e09-45fe-8683-c3b54d95448d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:12.982353 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-230a8e5d-3e09-45fe-8683-c3b54d95448d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd829cbd8678145668ea811e01093fcfc', 'request_id': 'req-230a8e5d-3e09-45fe-8683-c3b54d95448d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:12.990618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-230a8e5d-3e09-45fe-8683-c3b54d95448d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd829cbd8678145668ea811e01093fcfc', 'request_id': 'req-230a8e5d-3e09-45fe-8683-c3b54d95448d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:12.996374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-230a8e5d-3e09-45fe-8683-c3b54d95448d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd829cbd8678145668ea811e01093fcfc', 'request_id': 'req-230a8e5d-3e09-45fe-8683-c3b54d95448d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:13.008757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-230a8e5d-3e09-45fe-8683-c3b54d95448d admin admin] The token cache is being invalidate because project d829cbd8678145668ea811e01093fcfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:35:13.009036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-230a8e5d-3e09-45fe-8683-c3b54d95448d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:13.009713 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-230a8e5d-3e09-45fe-8683-c3b54d95448d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:13.011197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1189/2376] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:35:12 2022] DELETE /identity/v3/projects/d829cbd8678145668ea811e01093fcfc => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:13.016175 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca1ad2-8e7c-4a64-bfc9-e04012616d5e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:13.016466 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca1ad2-8e7c-4a64-bfc9-e04012616d5e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:13.016747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aca1ad2-8e7c-4a64-bfc9-e04012616d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:13.036188 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3aca1ad2-8e7c-4a64-bfc9-e04012616d5e None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:13.037429 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3aca1ad2-8e7c-4a64-bfc9-e04012616d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:13.094880 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1188/2377] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:13.103075 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e2232-e4c5-454f-b701-97b38b0b38fc None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:13.103347 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e2232-e4c5-454f-b701-97b38b0b38fc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:13.103622 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2e2232-e4c5-454f-b701-97b38b0b38fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:13.123188 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ee2e2232-e4c5-454f-b701-97b38b0b38fc None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:13.124368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ee2e2232-e4c5-454f-b701-97b38b0b38fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:13.134368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1190/2378] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:13.138697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca1ad2-8e7c-4a64-bfc9-e04012616d5e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:13.148690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aca1ad2-8e7c-4a64-bfc9-e04012616d5e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:13.153476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ec2f1-0ce4-4e10-b162-827856377166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:13.154152 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ec2f1-0ce4-4e10-b162-827856377166 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:13.154432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ec2f1-0ce4-4e10-b162-827856377166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:13.154704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3ec2f1-0ce4-4e10-b162-827856377166 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:13.155401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ec2f1-0ce4-4e10-b162-827856377166 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:35:13.155701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ec2f1-0ce4-4e10-b162-827856377166 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:13.157292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3ec2f1-0ce4-4e10-b162-827856377166 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:13.169776 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1189/2379] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:35:13.174065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e2232-e4c5-454f-b701-97b38b0b38fc None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:13.183506 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2e2232-e4c5-454f-b701-97b38b0b38fc None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:13.188095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1851da-16d2-4e6c-914c-60e995296282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:13.188698 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1851da-16d2-4e6c-914c-60e995296282 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:13.188984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1851da-16d2-4e6c-914c-60e995296282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:13.189257 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1851da-16d2-4e6c-914c-60e995296282 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:13.189942 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1851da-16d2-4e6c-914c-60e995296282 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:13.192328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1851da-16d2-4e6c-914c-60e995296282 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:13.196769 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:13.221920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1191/2380] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:35:13 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:13.226568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ec2f1-0ce4-4e10-b162-827856377166 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:13.236418 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3ec2f1-0ce4-4e10-b162-827856377166 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:13.280844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:13.294349 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75798ad8-9b02-4555-9502-7a0a4424c34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:13.294968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75798ad8-9b02-4555-9502-7a0a4424c34c admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:13.295248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75798ad8-9b02-4555-9502-7a0a4424c34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:13.295517 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75798ad8-9b02-4555-9502-7a0a4424c34c admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:13.296179 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75798ad8-9b02-4555-9502-7a0a4424c34c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:13.298113 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75798ad8-9b02-4555-9502-7a0a4424c34c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:13.318689 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1190/2381] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:35:13 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:13.323017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1851da-16d2-4e6c-914c-60e995296282 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:13.333869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1851da-16d2-4e6c-914c-60e995296282 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:13.338471 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a9b5e-a6cb-4443-bb7c-da09f35a0f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:13.339077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a9b5e-a6cb-4443-bb7c-da09f35a0f7f admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:13.339362 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a9b5e-a6cb-4443-bb7c-da09f35a0f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:13.339637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626a9b5e-a6cb-4443-bb7c-da09f35a0f7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:13.340330 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a9b5e-a6cb-4443-bb7c-da09f35a0f7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:13.341975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626a9b5e-a6cb-4443-bb7c-da09f35a0f7f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:13.352030 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1192/2382] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:13.355747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75798ad8-9b02-4555-9502-7a0a4424c34c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:13.365978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75798ad8-9b02-4555-9502-7a0a4424c34c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:13.370674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23196ed-cc5f-4dab-864d-fe94f23b6f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:13.371338 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23196ed-cc5f-4dab-864d-fe94f23b6f99 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:13.371658 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23196ed-cc5f-4dab-864d-fe94f23b6f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:13.371960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23196ed-cc5f-4dab-864d-fe94f23b6f99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:13.372656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23196ed-cc5f-4dab-864d-fe94f23b6f99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:13.374649 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23196ed-cc5f-4dab-864d-fe94f23b6f99 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:13.386138 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1191/2383] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:13.390262 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a9b5e-a6cb-4443-bb7c-da09f35a0f7f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:13.401551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626a9b5e-a6cb-4443-bb7c-da09f35a0f7f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:13.406193 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a1303-bc62-49ef-8b64-4f47c5204397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:13.406848 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a1303-bc62-49ef-8b64-4f47c5204397 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:13.407184 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a1303-bc62-49ef-8b64-4f47c5204397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:13.407463 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a1303-bc62-49ef-8b64-4f47c5204397 admin admin] PATH_INFO: `/v3/projects/846fa4f81b34443a8ecaf226cc45c0a8/users/a029e6506d774bafbf0288479e14f794/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:13.422202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a1303-bc62-49ef-8b64-4f47c5204397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=846fa4f81b34443a8ecaf226cc45c0a8, user_id=a029e6506d774bafbf0288479e14f794, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:13.424884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191a1303-bc62-49ef-8b64-4f47c5204397 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:13.437493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1193/2384] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:35:13 2022] PUT /identity/v3/projects/846fa4f81b34443a8ecaf226cc45c0a8/users/a029e6506d774bafbf0288479e14f794/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:13.449408 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dfefb-9c9d-4be9-b320-41545ab0c178 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:13.449723 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dfefb-9c9d-4be9-b320-41545ab0c178 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:13.450023 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dfefb-9c9d-4be9-b320-41545ab0c178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:13.477546 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3b5dfefb-9c9d-4be9-b320-41545ab0c178 None None] MFA Rules not processed for user `a029e6506d774bafbf0288479e14f794`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:13.478756 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3b5dfefb-9c9d-4be9-b320-41545ab0c178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:13.542377 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1192/2385] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:13 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:35:13.546793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a1303-bc62-49ef-8b64-4f47c5204397 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:13.607209 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a1303-bc62-49ef-8b64-4f47c5204397 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:13.611731 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d28f6-1606-4833-82a6-09bbc7693202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:13.612334 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d28f6-1606-4833-82a6-09bbc7693202 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:13.612616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d28f6-1606-4833-82a6-09bbc7693202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:13.612889 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d28f6-1606-4833-82a6-09bbc7693202 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:13.613548 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d28f6-1606-4833-82a6-09bbc7693202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:13.615925 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d28f6-1606-4833-82a6-09bbc7693202 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:13.642226 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1194/2386] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:35:13 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:13.646298 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dfefb-9c9d-4be9-b320-41545ab0c178 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:13.657052 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dfefb-9c9d-4be9-b320-41545ab0c178 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:13.712381 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee623472-332d-4126-8cf4-b0175c28a03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:13.713001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee623472-332d-4126-8cf4-b0175c28a03d admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:13.713300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee623472-332d-4126-8cf4-b0175c28a03d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:13.713597 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee623472-332d-4126-8cf4-b0175c28a03d admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:13.714263 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee623472-332d-4126-8cf4-b0175c28a03d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:13.716223 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee623472-332d-4126-8cf4-b0175c28a03d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:13.736602 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1193/2387] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:35:13 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:13.741319 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d28f6-1606-4833-82a6-09bbc7693202 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:13.751898 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d28f6-1606-4833-82a6-09bbc7693202 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:13.757116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed877cee-a82b-4c80-8361-4f47d013373a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:13.757820 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed877cee-a82b-4c80-8361-4f47d013373a admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:13.758106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed877cee-a82b-4c80-8361-4f47d013373a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:13.758381 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed877cee-a82b-4c80-8361-4f47d013373a admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:13.759078 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed877cee-a82b-4c80-8361-4f47d013373a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:13.760678 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed877cee-a82b-4c80-8361-4f47d013373a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:13.771694 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1195/2388] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:35:13.775777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee623472-332d-4126-8cf4-b0175c28a03d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:13.786704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee623472-332d-4126-8cf4-b0175c28a03d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:13.791069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1fa02-5e72-4d86-a60a-0e2bd9a4e18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:13.791721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1fa02-5e72-4d86-a60a-0e2bd9a4e18d admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:13.792001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1fa02-5e72-4d86-a60a-0e2bd9a4e18d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:13.792270 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1fa02-5e72-4d86-a60a-0e2bd9a4e18d admin admin] PATH_INFO: `/v3/projects/4472f9bf72e041f09128fa999d2ab4de/users/cecd34565eb349559dbde360fd5e09ab/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:13.808236 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae1fa02-5e72-4d86-a60a-0e2bd9a4e18d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4472f9bf72e041f09128fa999d2ab4de, user_id=cecd34565eb349559dbde360fd5e09ab, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:13.811097 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae1fa02-5e72-4d86-a60a-0e2bd9a4e18d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:13.825195 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1194/2389] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:35:13 2022] PUT /identity/v3/projects/4472f9bf72e041f09128fa999d2ab4de/users/cecd34565eb349559dbde360fd5e09ab/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:13.829820 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed877cee-a82b-4c80-8361-4f47d013373a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:13.841545 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed877cee-a82b-4c80-8361-4f47d013373a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:13.903255 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c29aaf-95f2-4abb-923a-3ee81cf01c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:13.903816 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c29aaf-95f2-4abb-923a-3ee81cf01c9b admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:13.904607 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c29aaf-95f2-4abb-923a-3ee81cf01c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:13.904607 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c29aaf-95f2-4abb-923a-3ee81cf01c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:13.905154 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c29aaf-95f2-4abb-923a-3ee81cf01c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:13.906830 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c29aaf-95f2-4abb-923a-3ee81cf01c9b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:13.918136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1196/2390] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:13.922109 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1fa02-5e72-4d86-a60a-0e2bd9a4e18d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:13.934274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1fa02-5e72-4d86-a60a-0e2bd9a4e18d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:13.949290 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749740e9-c3d2-4064-8622-c361c721dd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:13.950146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749740e9-c3d2-4064-8622-c361c721dd92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:13.950499 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749740e9-c3d2-4064-8622-c361c721dd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:13.950861 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749740e9-c3d2-4064-8622-c361c721dd92 admin admin] PATH_INFO: `/v3/projects/4472f9bf72e041f09128fa999d2ab4de/users/cecd34565eb349559dbde360fd5e09ab/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:13.952936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749740e9-c3d2-4064-8622-c361c721dd92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4472f9bf72e041f09128fa999d2ab4de, user_id=cecd34565eb349559dbde360fd5e09ab, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:13.955999 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749740e9-c3d2-4064-8622-c361c721dd92 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:13.969586 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1195/2391] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:35:13 2022] PUT /identity/v3/projects/4472f9bf72e041f09128fa999d2ab4de/users/cecd34565eb349559dbde360fd5e09ab/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:35:13.979819 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b35fe0-b4cf-4c1b-8a0c-4f6835d626f4 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:13.980123 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b35fe0-b4cf-4c1b-8a0c-4f6835d626f4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:13.980385 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b35fe0-b4cf-4c1b-8a0c-4f6835d626f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:14.018341 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-14b35fe0-b4cf-4c1b-8a0c-4f6835d626f4 None None] MFA Rules not processed for user `cecd34565eb349559dbde360fd5e09ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:14.019630 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-14b35fe0-b4cf-4c1b-8a0c-4f6835d626f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:14.033129 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:14.108030 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1197/2392] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:13 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:35:14.120540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749740e9-c3d2-4064-8622-c361c721dd92 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:14.122431 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-749740e9-c3d2-4064-8622-c361c721dd92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:14.133057 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:14.200133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749740e9-c3d2-4064-8622-c361c721dd92 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:14.211676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b388cd2-d8aa-4617-8963-a0b4520048c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:14.212465 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b388cd2-d8aa-4617-8963-a0b4520048c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:14.212703 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b388cd2-d8aa-4617-8963-a0b4520048c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:14.212978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b388cd2-d8aa-4617-8963-a0b4520048c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:14.222636 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b388cd2-d8aa-4617-8963-a0b4520048c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:14.224727 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b388cd2-d8aa-4617-8963-a0b4520048c3 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:14.236820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1196/2393] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:14 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:35:14.313005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b35fe0-b4cf-4c1b-8a0c-4f6835d626f4 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:14.323677 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b35fe0-b4cf-4c1b-8a0c-4f6835d626f4 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:14.327403 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c827ad27-bf95-4e87-8e73-7c3433a5ba06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:14.328088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827ad27-bf95-4e87-8e73-7c3433a5ba06 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:14.328439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827ad27-bf95-4e87-8e73-7c3433a5ba06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:14.328802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c827ad27-bf95-4e87-8e73-7c3433a5ba06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:14.388796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c827ad27-bf95-4e87-8e73-7c3433a5ba06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:14.390278 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c827ad27-bf95-4e87-8e73-7c3433a5ba06 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:14.400193 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1198/2394] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:14 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:35:17.135157 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b388cd2-d8aa-4617-8963-a0b4520048c3 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:17.196754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b388cd2-d8aa-4617-8963-a0b4520048c3 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:17.201277 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd56e9-7e9e-41de-8f8c-8121b52d56ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:17.201952 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd56e9-7e9e-41de-8f8c-8121b52d56ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:17.202238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd56e9-7e9e-41de-8f8c-8121b52d56ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:17.202515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd56e9-7e9e-41de-8f8c-8121b52d56ac admin admin] PATH_INFO: `/v3/users/a029e6506d774bafbf0288479e14f794` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:17.203677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd56e9-7e9e-41de-8f8c-8121b52d56ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=a029e6506d774bafbf0288479e14f794)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:17.205630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd56e9-7e9e-41de-8f8c-8121b52d56ac admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:17.250907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b5dd56e9-7e9e-41de-8f8c-8121b52d56ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a029e6506d774bafbf0288479e14f794'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:17.257975 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b5dd56e9-7e9e-41de-8f8c-8121b52d56ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a029e6506d774bafbf0288479e14f794'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:17.258647 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b5dd56e9-7e9e-41de-8f8c-8121b52d56ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a029e6506d774bafbf0288479e14f794'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:17.259246 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b5dd56e9-7e9e-41de-8f8c-8121b52d56ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a029e6506d774bafbf0288479e14f794'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:17.271424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b5dd56e9-7e9e-41de-8f8c-8121b52d56ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a029e6506d774bafbf0288479e14f794'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:17.284355 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1197/2395] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:35:17 2022] DELETE /identity/v3/users/a029e6506d774bafbf0288479e14f794 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:17.288881 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f6083-e25e-43ee-9d77-9a2fb82b1232 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:17.289379 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f6083-e25e-43ee-9d77-9a2fb82b1232 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:17.289478 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83f6083-e25e-43ee-9d77-9a2fb82b1232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:17.309102 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-c83f6083-e25e-43ee-9d77-9a2fb82b1232 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:17.310351 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c83f6083-e25e-43ee-9d77-9a2fb82b1232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:17.367228 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1199/2396] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:17.376810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd56e9-7e9e-41de-8f8c-8121b52d56ac admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:17.378478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b5dd56e9-7e9e-41de-8f8c-8121b52d56ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:17.410016 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:17.439435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd56e9-7e9e-41de-8f8c-8121b52d56ac admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:17.443735 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc4675-7301-4355-9baf-dcc552395673 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:17.444357 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc4675-7301-4355-9baf-dcc552395673 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:17.444638 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc4675-7301-4355-9baf-dcc552395673 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:17.444913 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc4675-7301-4355-9baf-dcc552395673 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:17.454274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc4675-7301-4355-9baf-dcc552395673 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:17.455730 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc4675-7301-4355-9baf-dcc552395673 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:17.465272 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1198/2397] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:35:17.756637 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f6083-e25e-43ee-9d77-9a2fb82b1232 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:17.758954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c83f6083-e25e-43ee-9d77-9a2fb82b1232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:17.775302 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83f6083-e25e-43ee-9d77-9a2fb82b1232 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:17.780184 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205ba91-7fa4-43bd-a94a-c9a7fd74b7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:17.780857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205ba91-7fa4-43bd-a94a-c9a7fd74b7f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:17.781143 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205ba91-7fa4-43bd-a94a-c9a7fd74b7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:17.781934 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e205ba91-7fa4-43bd-a94a-c9a7fd74b7f5 admin admin] PATH_INFO: `/v3/projects/846fa4f81b34443a8ecaf226cc45c0a8` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:17.782950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205ba91-7fa4-43bd-a94a-c9a7fd74b7f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=846fa4f81b34443a8ecaf226cc45c0a8)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:17.785013 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e205ba91-7fa4-43bd-a94a-c9a7fd74b7f5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:17.825422 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:17.834190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e205ba91-7fa4-43bd-a94a-c9a7fd74b7f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '846fa4f81b34443a8ecaf226cc45c0a8', 'request_id': 'req-e205ba91-7fa4-43bd-a94a-c9a7fd74b7f5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:17.841133 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e205ba91-7fa4-43bd-a94a-c9a7fd74b7f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '846fa4f81b34443a8ecaf226cc45c0a8', 'request_id': 'req-e205ba91-7fa4-43bd-a94a-c9a7fd74b7f5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:17.847191 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e205ba91-7fa4-43bd-a94a-c9a7fd74b7f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '846fa4f81b34443a8ecaf226cc45c0a8', 'request_id': 'req-e205ba91-7fa4-43bd-a94a-c9a7fd74b7f5'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:17.859290 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e205ba91-7fa4-43bd-a94a-c9a7fd74b7f5 admin admin] The token cache is being invalidate because project 846fa4f81b34443a8ecaf226cc45c0a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:35:17.859579 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e205ba91-7fa4-43bd-a94a-c9a7fd74b7f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:17.860274 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e205ba91-7fa4-43bd-a94a-c9a7fd74b7f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:17.862012 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1200/2398] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:35:17 2022] DELETE /identity/v3/projects/846fa4f81b34443a8ecaf226cc45c0a8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:17.866342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc4675-7301-4355-9baf-dcc552395673 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:17.877409 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc4675-7301-4355-9baf-dcc552395673 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:17.937228 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79128eba-92e6-4dd1-a701-640208f942e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:17.937899 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79128eba-92e6-4dd1-a701-640208f942e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:17.938206 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79128eba-92e6-4dd1-a701-640208f942e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:17.938484 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79128eba-92e6-4dd1-a701-640208f942e7 admin admin] PATH_INFO: `/v3/users/cecd34565eb349559dbde360fd5e09ab` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:17.939672 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79128eba-92e6-4dd1-a701-640208f942e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cecd34565eb349559dbde360fd5e09ab)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:17.942146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79128eba-92e6-4dd1-a701-640208f942e7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:17.995265 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-79128eba-92e6-4dd1-a701-640208f942e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cecd34565eb349559dbde360fd5e09ab'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:18.003805 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-79128eba-92e6-4dd1-a701-640208f942e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cecd34565eb349559dbde360fd5e09ab'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:18.005035 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-79128eba-92e6-4dd1-a701-640208f942e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cecd34565eb349559dbde360fd5e09ab'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:18.005715 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-79128eba-92e6-4dd1-a701-640208f942e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cecd34565eb349559dbde360fd5e09ab'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:18.020284 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-79128eba-92e6-4dd1-a701-640208f942e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cecd34565eb349559dbde360fd5e09ab'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:18.039686 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1199/2399] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:35:17 2022] DELETE /identity/v3/users/cecd34565eb349559dbde360fd5e09ab => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:18.327708 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205ba91-7fa4-43bd-a94a-c9a7fd74b7f5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:18.329245 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e205ba91-7fa4-43bd-a94a-c9a7fd74b7f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:18.348471 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e205ba91-7fa4-43bd-a94a-c9a7fd74b7f5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:18.405817 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a0c0c4-e83c-4e32-a509-febd2d8d081e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:18.406454 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0c0c4-e83c-4e32-a509-febd2d8d081e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:18.406733 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0c0c4-e83c-4e32-a509-febd2d8d081e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:18.407138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a0c0c4-e83c-4e32-a509-febd2d8d081e admin admin] PATH_INFO: `/v3/projects/4472f9bf72e041f09128fa999d2ab4de` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:18.408436 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0c0c4-e83c-4e32-a509-febd2d8d081e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4472f9bf72e041f09128fa999d2ab4de)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:18.410405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a0c0c4-e83c-4e32-a509-febd2d8d081e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:18.455910 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-67a0c0c4-e83c-4e32-a509-febd2d8d081e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4472f9bf72e041f09128fa999d2ab4de', 'request_id': 'req-67a0c0c4-e83c-4e32-a509-febd2d8d081e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:18.463756 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-67a0c0c4-e83c-4e32-a509-febd2d8d081e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4472f9bf72e041f09128fa999d2ab4de', 'request_id': 'req-67a0c0c4-e83c-4e32-a509-febd2d8d081e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:18.469332 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-67a0c0c4-e83c-4e32-a509-febd2d8d081e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4472f9bf72e041f09128fa999d2ab4de', 'request_id': 'req-67a0c0c4-e83c-4e32-a509-febd2d8d081e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:18.480829 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-67a0c0c4-e83c-4e32-a509-febd2d8d081e admin admin] The token cache is being invalidate because project 4472f9bf72e041f09128fa999d2ab4de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:35:18.481134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-67a0c0c4-e83c-4e32-a509-febd2d8d081e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:18.481791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-67a0c0c4-e83c-4e32-a509-febd2d8d081e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:18.483266 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1201/2400] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:35:18 2022] DELETE /identity/v3/projects/4472f9bf72e041f09128fa999d2ab4de => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:18.488407 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc902a4-7d31-4d49-8e27-49165c761020 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:18.488931 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc902a4-7d31-4d49-8e27-49165c761020 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:18.489017 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc902a4-7d31-4d49-8e27-49165c761020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:18.509065 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-edc902a4-7d31-4d49-8e27-49165c761020 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:18.510294 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-edc902a4-7d31-4d49-8e27-49165c761020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:18.554121 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:18.570650 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1200/2401] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:18.579822 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bf811-4015-447f-ab11-fca7d36ebd33 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:18.580045 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bf811-4015-447f-ab11-fca7d36ebd33 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:18.580320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140bf811-4015-447f-ab11-fca7d36ebd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:18.600786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-140bf811-4015-447f-ab11-fca7d36ebd33 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:18.602031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-140bf811-4015-447f-ab11-fca7d36ebd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:18.609790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1202/2402] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:18.614882 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc902a4-7d31-4d49-8e27-49165c761020 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:18.624643 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc902a4-7d31-4d49-8e27-49165c761020 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:18.629915 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188f0e1-ce3e-46ee-8f75-1114512f9669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:18.630520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188f0e1-ce3e-46ee-8f75-1114512f9669 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:18.630804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188f0e1-ce3e-46ee-8f75-1114512f9669 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:18.631078 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9188f0e1-ce3e-46ee-8f75-1114512f9669 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:18.631805 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188f0e1-ce3e-46ee-8f75-1114512f9669 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:35:18.632110 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188f0e1-ce3e-46ee-8f75-1114512f9669 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:18.633746 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9188f0e1-ce3e-46ee-8f75-1114512f9669 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:18.645509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1201/2403] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:35:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:35:18.649811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bf811-4015-447f-ab11-fca7d36ebd33 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:18.658852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140bf811-4015-447f-ab11-fca7d36ebd33 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:18.664169 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef14cb-1087-4965-a9ac-42e598442bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:18.664764 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef14cb-1087-4965-a9ac-42e598442bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:18.665040 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef14cb-1087-4965-a9ac-42e598442bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:18.665306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef14cb-1087-4965-a9ac-42e598442bd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:18.665997 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef14cb-1087-4965-a9ac-42e598442bd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:18.667873 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef14cb-1087-4965-a9ac-42e598442bd1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:18.694364 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1203/2404] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:35:18 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:18.698419 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188f0e1-ce3e-46ee-8f75-1114512f9669 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:18.708669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9188f0e1-ce3e-46ee-8f75-1114512f9669 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:18.769037 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2754032-03f7-43b6-a147-cc554667aef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:18.770195 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2754032-03f7-43b6-a147-cc554667aef6 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:18.770475 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2754032-03f7-43b6-a147-cc554667aef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:18.770770 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2754032-03f7-43b6-a147-cc554667aef6 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:18.771462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2754032-03f7-43b6-a147-cc554667aef6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:18.773375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2754032-03f7-43b6-a147-cc554667aef6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:18.796851 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1202/2405] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:35:18 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:35:18.801151 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef14cb-1087-4965-a9ac-42e598442bd1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:18.813432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef14cb-1087-4965-a9ac-42e598442bd1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:18.819053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22bcc20-4744-487f-8814-9e66f816905d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:18.819799 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bcc20-4744-487f-8814-9e66f816905d admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:18.820082 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bcc20-4744-487f-8814-9e66f816905d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:18.820373 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22bcc20-4744-487f-8814-9e66f816905d admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:18.821083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22bcc20-4744-487f-8814-9e66f816905d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:18.822832 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22bcc20-4744-487f-8814-9e66f816905d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:18.834939 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1204/2406] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:18.838918 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2754032-03f7-43b6-a147-cc554667aef6 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:18.850679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2754032-03f7-43b6-a147-cc554667aef6 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:18.855515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0811dc52-46f3-4eb3-95d2-acd65ce3bcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:18.856137 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811dc52-46f3-4eb3-95d2-acd65ce3bcea admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:18.856422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811dc52-46f3-4eb3-95d2-acd65ce3bcea admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:18.856699 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0811dc52-46f3-4eb3-95d2-acd65ce3bcea admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:18.857404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811dc52-46f3-4eb3-95d2-acd65ce3bcea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:18.859043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0811dc52-46f3-4eb3-95d2-acd65ce3bcea admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:18.869946 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1203/2407] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:35:18.874228 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22bcc20-4744-487f-8814-9e66f816905d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:18.885292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22bcc20-4744-487f-8814-9e66f816905d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:18.890030 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8825110-2716-432a-b922-721770814f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:18.890688 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8825110-2716-432a-b922-721770814f9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:18.890968 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8825110-2716-432a-b922-721770814f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:18.891243 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8825110-2716-432a-b922-721770814f9a admin admin] PATH_INFO: `/v3/projects/5bddcb6def5f46528f2178a60956d7b5/users/82c6144e5c1d435cb6e09a0b88d00b28/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:18.906992 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8825110-2716-432a-b922-721770814f9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bddcb6def5f46528f2178a60956d7b5, user_id=82c6144e5c1d435cb6e09a0b88d00b28, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:18.909670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8825110-2716-432a-b922-721770814f9a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:18.912662 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:18.924805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1205/2408] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:35:18 2022] PUT /identity/v3/projects/5bddcb6def5f46528f2178a60956d7b5/users/82c6144e5c1d435cb6e09a0b88d00b28/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:35:18.935125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e4e2c-7f05-4ac6-a60b-14b2464eca15 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:18.935415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e4e2c-7f05-4ac6-a60b-14b2464eca15 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:18.935692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2e4e2c-7f05-4ac6-a60b-14b2464eca15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:18.963993 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ec2e4e2c-7f05-4ac6-a60b-14b2464eca15 None None] MFA Rules not processed for user `82c6144e5c1d435cb6e09a0b88d00b28`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:18.965202 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ec2e4e2c-7f05-4ac6-a60b-14b2464eca15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:19.030471 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1204/2409] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:19.034941 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8825110-2716-432a-b922-721770814f9a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:19.094474 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8825110-2716-432a-b922-721770814f9a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:19.099793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466815cf-1415-47fd-bbeb-232c19e5bc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:19.100397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466815cf-1415-47fd-bbeb-232c19e5bc07 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:19.100677 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466815cf-1415-47fd-bbeb-232c19e5bc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:19.100951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466815cf-1415-47fd-bbeb-232c19e5bc07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:19.101641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466815cf-1415-47fd-bbeb-232c19e5bc07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:19.103526 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466815cf-1415-47fd-bbeb-232c19e5bc07 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:19.129907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1206/2410] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:35:19 2022] POST /identity/v3/projects => generated 369 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:19.134254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e4e2c-7f05-4ac6-a60b-14b2464eca15 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:19.146235 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2e4e2c-7f05-4ac6-a60b-14b2464eca15 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:19.189294 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:19.204080 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183ec3e-e432-4f63-9fa2-98784d740e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:19.204702 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183ec3e-e432-4f63-9fa2-98784d740e40 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:19.204984 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183ec3e-e432-4f63-9fa2-98784d740e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:19.205254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183ec3e-e432-4f63-9fa2-98784d740e40 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:19.205965 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183ec3e-e432-4f63-9fa2-98784d740e40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:19.209027 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183ec3e-e432-4f63-9fa2-98784d740e40 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:19.229029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1205/2411] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:35:19 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:19.233010 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466815cf-1415-47fd-bbeb-232c19e5bc07 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:19.243678 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466815cf-1415-47fd-bbeb-232c19e5bc07 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:19.248232 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2701443-d9a3-442f-8684-8a77d8ee389c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:19.248843 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2701443-d9a3-442f-8684-8a77d8ee389c admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:19.249122 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2701443-d9a3-442f-8684-8a77d8ee389c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:19.249398 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2701443-d9a3-442f-8684-8a77d8ee389c admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:19.250128 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2701443-d9a3-442f-8684-8a77d8ee389c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:19.251729 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2701443-d9a3-442f-8684-8a77d8ee389c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:19.263040 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1207/2412] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:19.267008 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183ec3e-e432-4f63-9fa2-98784d740e40 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:19.277405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183ec3e-e432-4f63-9fa2-98784d740e40 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:19.281957 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683f6039-f32b-4bc6-946d-d3a73e6b1d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:19.282601 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f6039-f32b-4bc6-946d-d3a73e6b1d5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:19.282883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f6039-f32b-4bc6-946d-d3a73e6b1d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:19.283155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f6039-f32b-4bc6-946d-d3a73e6b1d5b admin admin] PATH_INFO: `/v3/projects/34156c3bc6e54273bca4fdfdead54654/users/d3559640513a4884b0d425c57047b3a8/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:19.298138 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683f6039-f32b-4bc6-946d-d3a73e6b1d5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=34156c3bc6e54273bca4fdfdead54654, user_id=d3559640513a4884b0d425c57047b3a8, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:19.300777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683f6039-f32b-4bc6-946d-d3a73e6b1d5b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:19.313310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1206/2413] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:35:19 2022] PUT /identity/v3/projects/34156c3bc6e54273bca4fdfdead54654/users/d3559640513a4884b0d425c57047b3a8/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:19.317830 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2701443-d9a3-442f-8684-8a77d8ee389c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:19.328058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2701443-d9a3-442f-8684-8a77d8ee389c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:19.387376 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc67ca0-1d44-4cf5-bbd4-3d21ae8fe18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:19.388289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc67ca0-1d44-4cf5-bbd4-3d21ae8fe18b admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:19.388583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc67ca0-1d44-4cf5-bbd4-3d21ae8fe18b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:19.388935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc67ca0-1d44-4cf5-bbd4-3d21ae8fe18b admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:19.389704 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc67ca0-1d44-4cf5-bbd4-3d21ae8fe18b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:19.391306 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc67ca0-1d44-4cf5-bbd4-3d21ae8fe18b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:19.401982 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1208/2414] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:19.406139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683f6039-f32b-4bc6-946d-d3a73e6b1d5b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:19.416942 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683f6039-f32b-4bc6-946d-d3a73e6b1d5b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:19.421520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f035670-4341-4a56-af61-84801624aceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:19.422210 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f035670-4341-4a56-af61-84801624aceb admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:19.422493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f035670-4341-4a56-af61-84801624aceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:19.422766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f035670-4341-4a56-af61-84801624aceb admin admin] PATH_INFO: `/v3/projects/34156c3bc6e54273bca4fdfdead54654/users/d3559640513a4884b0d425c57047b3a8/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:19.424716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f035670-4341-4a56-af61-84801624aceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=34156c3bc6e54273bca4fdfdead54654, user_id=d3559640513a4884b0d425c57047b3a8, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:19.428039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f035670-4341-4a56-af61-84801624aceb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:19.438270 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1207/2415] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:35:19 2022] PUT /identity/v3/projects/34156c3bc6e54273bca4fdfdead54654/users/d3559640513a4884b0d425c57047b3a8/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:19.448990 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ac649-ee6b-47fc-a556-e4b8a60d90aa None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:19.449285 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ac649-ee6b-47fc-a556-e4b8a60d90aa None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:19.449596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ac649-ee6b-47fc-a556-e4b8a60d90aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:19.478769 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-314ac649-ee6b-47fc-a556-e4b8a60d90aa None None] MFA Rules not processed for user `d3559640513a4884b0d425c57047b3a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:19.479988 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-314ac649-ee6b-47fc-a556-e4b8a60d90aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:19.533677 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:19.555299 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1209/2416] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:19 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:19.566195 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f035670-4341-4a56-af61-84801624aceb admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:19.568266 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4f035670-4341-4a56-af61-84801624aceb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:19.627767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f035670-4341-4a56-af61-84801624aceb admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:19.631644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e7c38-3e44-4078-989e-45da63396ea9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:19.632275 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e7c38-3e44-4078-989e-45da63396ea9 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:19.632541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e7c38-3e44-4078-989e-45da63396ea9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:19.632815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e7c38-3e44-4078-989e-45da63396ea9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:19.642502 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e7c38-3e44-4078-989e-45da63396ea9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:19.643862 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1e7c38-3e44-4078-989e-45da63396ea9 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:19.654202 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1208/2417] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:19 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:35:19.699937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ac649-ee6b-47fc-a556-e4b8a60d90aa None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:19.711098 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ac649-ee6b-47fc-a556-e4b8a60d90aa None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:19.716038 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78a0f7b-8332-4a13-95e9-b516664bed71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:19.717811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a0f7b-8332-4a13-95e9-b516664bed71 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:19.718137 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a0f7b-8332-4a13-95e9-b516664bed71 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:19.718436 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78a0f7b-8332-4a13-95e9-b516664bed71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:19.780644 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a0f7b-8332-4a13-95e9-b516664bed71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:19.782136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78a0f7b-8332-4a13-95e9-b516664bed71 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:19.791845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1210/2418] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:19 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:35:23.231582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e7c38-3e44-4078-989e-45da63396ea9 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:23.302337 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e7c38-3e44-4078-989e-45da63396ea9 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:23.307332 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06633ac9-c41f-4adc-a5f4-471e5f8dd59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:23.307972 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06633ac9-c41f-4adc-a5f4-471e5f8dd59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:23.308251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06633ac9-c41f-4adc-a5f4-471e5f8dd59c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:23.308523 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06633ac9-c41f-4adc-a5f4-471e5f8dd59c admin admin] PATH_INFO: `/v3/users/82c6144e5c1d435cb6e09a0b88d00b28` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:23.309719 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06633ac9-c41f-4adc-a5f4-471e5f8dd59c admin admin] RBAC: Authorizing `identity:delete_user(user_id=82c6144e5c1d435cb6e09a0b88d00b28)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:23.311660 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06633ac9-c41f-4adc-a5f4-471e5f8dd59c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:23.360354 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:23.364135 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-06633ac9-c41f-4adc-a5f4-471e5f8dd59c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82c6144e5c1d435cb6e09a0b88d00b28'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:23.376351 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-06633ac9-c41f-4adc-a5f4-471e5f8dd59c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82c6144e5c1d435cb6e09a0b88d00b28'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:23.377314 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-06633ac9-c41f-4adc-a5f4-471e5f8dd59c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82c6144e5c1d435cb6e09a0b88d00b28'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:23.377959 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-06633ac9-c41f-4adc-a5f4-471e5f8dd59c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82c6144e5c1d435cb6e09a0b88d00b28'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:23.394118 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-06633ac9-c41f-4adc-a5f4-471e5f8dd59c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82c6144e5c1d435cb6e09a0b88d00b28'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:23.407151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1209/2419] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:35:23 2022] DELETE /identity/v3/users/82c6144e5c1d435cb6e09a0b88d00b28 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:23.412361 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29e1b6-c107-4d1c-9ffb-94383708af05 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:23.412892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29e1b6-c107-4d1c-9ffb-94383708af05 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:23.412892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29e1b6-c107-4d1c-9ffb-94383708af05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:23.431529 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8f29e1b6-c107-4d1c-9ffb-94383708af05 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:23.432715 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8f29e1b6-c107-4d1c-9ffb-94383708af05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:23.493150 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1211/2420] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:35:23.503305 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06633ac9-c41f-4adc-a5f4-471e5f8dd59c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:23.504766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-06633ac9-c41f-4adc-a5f4-471e5f8dd59c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:23.565135 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06633ac9-c41f-4adc-a5f4-471e5f8dd59c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:23.568856 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec816e70-25e3-49f6-b657-4a4bb2f59cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:23.569512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec816e70-25e3-49f6-b657-4a4bb2f59cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:23.569825 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec816e70-25e3-49f6-b657-4a4bb2f59cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:23.570105 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec816e70-25e3-49f6-b657-4a4bb2f59cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:23.579863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec816e70-25e3-49f6-b657-4a4bb2f59cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:23.581304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec816e70-25e3-49f6-b657-4a4bb2f59cf2 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:23.593237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1210/2421] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:35:23.896059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f29e1b6-c107-4d1c-9ffb-94383708af05 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:23.898114 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8f29e1b6-c107-4d1c-9ffb-94383708af05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:23.935600 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f29e1b6-c107-4d1c-9ffb-94383708af05 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:23.941031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a97115-7a80-4bd3-88e3-ea19064b9a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:23.942261 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a97115-7a80-4bd3-88e3-ea19064b9a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:23.942769 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a97115-7a80-4bd3-88e3-ea19064b9a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:23.942886 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a97115-7a80-4bd3-88e3-ea19064b9a74 admin admin] PATH_INFO: `/v3/projects/5bddcb6def5f46528f2178a60956d7b5` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:23.944047 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a97115-7a80-4bd3-88e3-ea19064b9a74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bddcb6def5f46528f2178a60956d7b5)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:23.946033 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a97115-7a80-4bd3-88e3-ea19064b9a74 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:24.018358 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-96a97115-7a80-4bd3-88e3-ea19064b9a74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bddcb6def5f46528f2178a60956d7b5', 'request_id': 'req-96a97115-7a80-4bd3-88e3-ea19064b9a74'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:24.029106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-96a97115-7a80-4bd3-88e3-ea19064b9a74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bddcb6def5f46528f2178a60956d7b5', 'request_id': 'req-96a97115-7a80-4bd3-88e3-ea19064b9a74'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:24.041183 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-96a97115-7a80-4bd3-88e3-ea19064b9a74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bddcb6def5f46528f2178a60956d7b5', 'request_id': 'req-96a97115-7a80-4bd3-88e3-ea19064b9a74'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:24.057213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-96a97115-7a80-4bd3-88e3-ea19064b9a74 admin admin] The token cache is being invalidate because project 5bddcb6def5f46528f2178a60956d7b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:35:24.057657 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-96a97115-7a80-4bd3-88e3-ea19064b9a74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:24.058400 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-96a97115-7a80-4bd3-88e3-ea19064b9a74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:24.065016 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1212/2422] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:35:23 2022] DELETE /identity/v3/projects/5bddcb6def5f46528f2178a60956d7b5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:24.066382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec816e70-25e3-49f6-b657-4a4bb2f59cf2 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:24.077753 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec816e70-25e3-49f6-b657-4a4bb2f59cf2 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:24.140209 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcd2c19-cd59-41b8-af2d-63154683fdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:24.142022 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd2c19-cd59-41b8-af2d-63154683fdbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:24.142022 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd2c19-cd59-41b8-af2d-63154683fdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:24.142022 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd2c19-cd59-41b8-af2d-63154683fdbd admin admin] PATH_INFO: `/v3/users/d3559640513a4884b0d425c57047b3a8` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:24.143052 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd2c19-cd59-41b8-af2d-63154683fdbd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3559640513a4884b0d425c57047b3a8)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:24.146068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcd2c19-cd59-41b8-af2d-63154683fdbd admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:24.206044 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3fcd2c19-cd59-41b8-af2d-63154683fdbd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3559640513a4884b0d425c57047b3a8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:24.214693 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3fcd2c19-cd59-41b8-af2d-63154683fdbd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3559640513a4884b0d425c57047b3a8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:24.215403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3fcd2c19-cd59-41b8-af2d-63154683fdbd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3559640513a4884b0d425c57047b3a8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:24.216038 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3fcd2c19-cd59-41b8-af2d-63154683fdbd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3559640513a4884b0d425c57047b3a8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:24.225424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:24.237269 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3fcd2c19-cd59-41b8-af2d-63154683fdbd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3559640513a4884b0d425c57047b3a8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:24.251396 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1211/2423] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:35:24 2022] DELETE /identity/v3/users/d3559640513a4884b0d425c57047b3a8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:24.574969 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a97115-7a80-4bd3-88e3-ea19064b9a74 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:24.576699 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-96a97115-7a80-4bd3-88e3-ea19064b9a74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:24.592096 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a97115-7a80-4bd3-88e3-ea19064b9a74 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:24.603226 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:24.661246 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14b8a0-8ea0-4e23-a48f-20840b2cbc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:24.661861 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14b8a0-8ea0-4e23-a48f-20840b2cbc8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:24.662143 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14b8a0-8ea0-4e23-a48f-20840b2cbc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:24.662418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14b8a0-8ea0-4e23-a48f-20840b2cbc8c admin admin] PATH_INFO: `/v3/projects/34156c3bc6e54273bca4fdfdead54654` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:24.663613 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14b8a0-8ea0-4e23-a48f-20840b2cbc8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=34156c3bc6e54273bca4fdfdead54654)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:24.665514 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14b8a0-8ea0-4e23-a48f-20840b2cbc8c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:24.711796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3c14b8a0-8ea0-4e23-a48f-20840b2cbc8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34156c3bc6e54273bca4fdfdead54654', 'request_id': 'req-3c14b8a0-8ea0-4e23-a48f-20840b2cbc8c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:24.718367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3c14b8a0-8ea0-4e23-a48f-20840b2cbc8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34156c3bc6e54273bca4fdfdead54654', 'request_id': 'req-3c14b8a0-8ea0-4e23-a48f-20840b2cbc8c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:24.724232 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3c14b8a0-8ea0-4e23-a48f-20840b2cbc8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34156c3bc6e54273bca4fdfdead54654', 'request_id': 'req-3c14b8a0-8ea0-4e23-a48f-20840b2cbc8c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:24.735183 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3c14b8a0-8ea0-4e23-a48f-20840b2cbc8c admin admin] The token cache is being invalidate because project 34156c3bc6e54273bca4fdfdead54654 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:35:24.735494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3c14b8a0-8ea0-4e23-a48f-20840b2cbc8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:24.736122 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3c14b8a0-8ea0-4e23-a48f-20840b2cbc8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:24.737652 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1213/2424] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:35:24 2022] DELETE /identity/v3/projects/34156c3bc6e54273bca4fdfdead54654 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:24.743230 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621601c-1c79-4d83-ba8d-8d2528b69ab3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:24.743564 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621601c-1c79-4d83-ba8d-8d2528b69ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:24.743839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4621601c-1c79-4d83-ba8d-8d2528b69ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:24.765829 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4621601c-1c79-4d83-ba8d-8d2528b69ab3 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:24.767021 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4621601c-1c79-4d83-ba8d-8d2528b69ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:24.837840 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1212/2425] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:24.845750 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adbeea-8810-448b-af0d-e345c8ac9a25 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:24.846042 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adbeea-8810-448b-af0d-e345c8ac9a25 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:24.846319 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41adbeea-8810-448b-af0d-e345c8ac9a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:24.866849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-41adbeea-8810-448b-af0d-e345c8ac9a25 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:24.868061 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-41adbeea-8810-448b-af0d-e345c8ac9a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:24.876964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1214/2426] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:24.881801 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621601c-1c79-4d83-ba8d-8d2528b69ab3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:24.891661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4621601c-1c79-4d83-ba8d-8d2528b69ab3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:24.896195 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1716a-d8cf-4381-ab3b-b65daa5518eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:24.896800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1716a-d8cf-4381-ab3b-b65daa5518eb admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:24.897078 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1716a-d8cf-4381-ab3b-b65daa5518eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:24.897346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1716a-d8cf-4381-ab3b-b65daa5518eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:24.898078 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d1716a-d8cf-4381-ab3b-b65daa5518eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:35:24.898379 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d1716a-d8cf-4381-ab3b-b65daa5518eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:24.899974 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d1716a-d8cf-4381-ab3b-b65daa5518eb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:24.911681 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1213/2427] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:35:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:35:24.915899 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41adbeea-8810-448b-af0d-e345c8ac9a25 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:24.925260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41adbeea-8810-448b-af0d-e345c8ac9a25 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:24.930737 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bccd16-3a14-4dfa-a3b3-a307faefebbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:24.931345 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bccd16-3a14-4dfa-a3b3-a307faefebbb admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:24.931630 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bccd16-3a14-4dfa-a3b3-a307faefebbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:24.931903 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bccd16-3a14-4dfa-a3b3-a307faefebbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:24.932573 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bccd16-3a14-4dfa-a3b3-a307faefebbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:24.934522 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bccd16-3a14-4dfa-a3b3-a307faefebbb admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:24.961994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1215/2428] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:35:24 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:24.966108 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1716a-d8cf-4381-ab3b-b65daa5518eb admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:24.976204 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1716a-d8cf-4381-ab3b-b65daa5518eb admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:25.033774 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb9333-e13e-4027-991d-54e41eccb733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:25.034378 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb9333-e13e-4027-991d-54e41eccb733 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:25.034654 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb9333-e13e-4027-991d-54e41eccb733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:25.034960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eb9333-e13e-4027-991d-54e41eccb733 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:25.035761 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eb9333-e13e-4027-991d-54e41eccb733 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:25.037763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eb9333-e13e-4027-991d-54e41eccb733 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:25.059147 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1214/2429] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:35:24 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:25.062857 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bccd16-3a14-4dfa-a3b3-a307faefebbb admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:25.074316 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bccd16-3a14-4dfa-a3b3-a307faefebbb admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:25.078878 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d90398-cf61-4499-b9f2-c9d475e8c100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:25.079486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d90398-cf61-4499-b9f2-c9d475e8c100 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:25.079767 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d90398-cf61-4499-b9f2-c9d475e8c100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:25.080040 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d90398-cf61-4499-b9f2-c9d475e8c100 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:25.080730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d90398-cf61-4499-b9f2-c9d475e8c100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:25.082955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d90398-cf61-4499-b9f2-c9d475e8c100 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:25.093304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1216/2430] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:25.097326 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb9333-e13e-4027-991d-54e41eccb733 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:25.107562 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eb9333-e13e-4027-991d-54e41eccb733 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:25.116511 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06d120-0ffa-4b67-a64b-4f3ce9b55d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:25.120186 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06d120-0ffa-4b67-a64b-4f3ce9b55d3c admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:25.120501 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06d120-0ffa-4b67-a64b-4f3ce9b55d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:25.120791 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06d120-0ffa-4b67-a64b-4f3ce9b55d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:25.121519 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b06d120-0ffa-4b67-a64b-4f3ce9b55d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:25.123160 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b06d120-0ffa-4b67-a64b-4f3ce9b55d3c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:25.140874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1215/2431] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:25.145416 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d90398-cf61-4499-b9f2-c9d475e8c100 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:25.156352 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d90398-cf61-4499-b9f2-c9d475e8c100 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:25.161962 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2f4c8-a80c-4e1a-b27b-1518939f0853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:25.162602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2f4c8-a80c-4e1a-b27b-1518939f0853 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:25.162881 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2f4c8-a80c-4e1a-b27b-1518939f0853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:25.163200 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a2f4c8-a80c-4e1a-b27b-1518939f0853 admin admin] PATH_INFO: `/v3/projects/4acdcdd4c28d417f8e373bd1c7487413/users/ddf962750dde4852a841b5e6cf2bfc14/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:25.183052 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2f4c8-a80c-4e1a-b27b-1518939f0853 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4acdcdd4c28d417f8e373bd1c7487413, user_id=ddf962750dde4852a841b5e6cf2bfc14, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:25.185788 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a2f4c8-a80c-4e1a-b27b-1518939f0853 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:25.200971 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1217/2432] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:35:25 2022] PUT /identity/v3/projects/4acdcdd4c28d417f8e373bd1c7487413/users/ddf962750dde4852a841b5e6cf2bfc14/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:35:25.211904 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673861e-91b6-4a46-96e8-04c4bf7b67c3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:25.212374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673861e-91b6-4a46-96e8-04c4bf7b67c3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:25.212720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673861e-91b6-4a46-96e8-04c4bf7b67c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:25.252950 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e673861e-91b6-4a46-96e8-04c4bf7b67c3 None None] MFA Rules not processed for user `ddf962750dde4852a841b5e6cf2bfc14`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:25.254410 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e673861e-91b6-4a46-96e8-04c4bf7b67c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:25.294040 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:25.321920 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1216/2433] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:25.326534 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2f4c8-a80c-4e1a-b27b-1518939f0853 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:25.367553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:25.388934 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a2f4c8-a80c-4e1a-b27b-1518939f0853 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:25.393488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0a85c-18f0-4bc8-94ef-eec2ec8997a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:25.394120 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0a85c-18f0-4bc8-94ef-eec2ec8997a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:25.394437 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0a85c-18f0-4bc8-94ef-eec2ec8997a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:25.394712 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0a85c-18f0-4bc8-94ef-eec2ec8997a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:25.395367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0a85c-18f0-4bc8-94ef-eec2ec8997a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:25.397284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0a85c-18f0-4bc8-94ef-eec2ec8997a9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:25.423208 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1218/2434] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:35:25 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:25.427808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673861e-91b6-4a46-96e8-04c4bf7b67c3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:25.438798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673861e-91b6-4a46-96e8-04c4bf7b67c3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:25.495647 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf09052-02c4-4ddb-91d7-b81786d0f9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:25.496130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf09052-02c4-4ddb-91d7-b81786d0f9c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:25.496411 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf09052-02c4-4ddb-91d7-b81786d0f9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:25.496684 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf09052-02c4-4ddb-91d7-b81786d0f9c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:25.497391 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf09052-02c4-4ddb-91d7-b81786d0f9c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:25.499353 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf09052-02c4-4ddb-91d7-b81786d0f9c1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:25.521796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1217/2435] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:35:25 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:25.525945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0a85c-18f0-4bc8-94ef-eec2ec8997a9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:25.536916 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0a85c-18f0-4bc8-94ef-eec2ec8997a9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:25.541439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07116e25-1360-4ce6-91a4-0c4a005f2fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:25.542084 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07116e25-1360-4ce6-91a4-0c4a005f2fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:25.542364 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07116e25-1360-4ce6-91a4-0c4a005f2fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:25.542636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07116e25-1360-4ce6-91a4-0c4a005f2fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:25.543349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07116e25-1360-4ce6-91a4-0c4a005f2fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:25.545092 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07116e25-1360-4ce6-91a4-0c4a005f2fe5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:25.556393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1219/2436] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:25.560600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf09052-02c4-4ddb-91d7-b81786d0f9c1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:25.570997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf09052-02c4-4ddb-91d7-b81786d0f9c1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:25.575743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d806d8b-532a-4fdf-9ae8-b2e1d6df4b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:25.576405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d806d8b-532a-4fdf-9ae8-b2e1d6df4b6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:25.576690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d806d8b-532a-4fdf-9ae8-b2e1d6df4b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:25.576972 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d806d8b-532a-4fdf-9ae8-b2e1d6df4b6c admin admin] PATH_INFO: `/v3/projects/b71c38d365524f079227e87f0d84e193/users/acbd5bd3de2d4161aaafe9b05d7bb209/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:25.602440 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d806d8b-532a-4fdf-9ae8-b2e1d6df4b6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b71c38d365524f079227e87f0d84e193, user_id=acbd5bd3de2d4161aaafe9b05d7bb209, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:25.605127 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d806d8b-532a-4fdf-9ae8-b2e1d6df4b6c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:25.624308 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1218/2437] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:35:25 2022] PUT /identity/v3/projects/b71c38d365524f079227e87f0d84e193/users/acbd5bd3de2d4161aaafe9b05d7bb209/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:25.625096 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07116e25-1360-4ce6-91a4-0c4a005f2fe5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:25.638778 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07116e25-1360-4ce6-91a4-0c4a005f2fe5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:25.700555 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3839d41d-c1f8-4696-ac17-a5076e46f228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:25.701174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839d41d-c1f8-4696-ac17-a5076e46f228 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:25.701484 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839d41d-c1f8-4696-ac17-a5076e46f228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:25.701993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3839d41d-c1f8-4696-ac17-a5076e46f228 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:25.702695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839d41d-c1f8-4696-ac17-a5076e46f228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:25.704296 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3839d41d-c1f8-4696-ac17-a5076e46f228 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:25.717012 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1220/2438] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:35:25.721227 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d806d8b-532a-4fdf-9ae8-b2e1d6df4b6c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:25.733049 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d806d8b-532a-4fdf-9ae8-b2e1d6df4b6c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:25.738029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba2baa-3844-4e69-b195-ad547b984181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:25.738834 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba2baa-3844-4e69-b195-ad547b984181 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:25.739069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba2baa-3844-4e69-b195-ad547b984181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:25.739345 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba2baa-3844-4e69-b195-ad547b984181 admin admin] PATH_INFO: `/v3/projects/b71c38d365524f079227e87f0d84e193/users/acbd5bd3de2d4161aaafe9b05d7bb209/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:25.741287 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba2baa-3844-4e69-b195-ad547b984181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b71c38d365524f079227e87f0d84e193, user_id=acbd5bd3de2d4161aaafe9b05d7bb209, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:25.744062 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba2baa-3844-4e69-b195-ad547b984181 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:25.755216 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1219/2439] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:35:25 2022] PUT /identity/v3/projects/b71c38d365524f079227e87f0d84e193/users/acbd5bd3de2d4161aaafe9b05d7bb209/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:25.766436 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165a775-38c7-4c36-ba4d-816f5b6db668 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:25.766723 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165a775-38c7-4c36-ba4d-816f5b6db668 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:25.766996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8165a775-38c7-4c36-ba4d-816f5b6db668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:25.796293 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8165a775-38c7-4c36-ba4d-816f5b6db668 None None] MFA Rules not processed for user `acbd5bd3de2d4161aaafe9b05d7bb209`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:25.797621 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8165a775-38c7-4c36-ba4d-816f5b6db668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:25.871125 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1221/2440] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:25 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:25.891032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba2baa-3844-4e69-b195-ad547b984181 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:25.893145 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-baba2baa-3844-4e69-b195-ad547b984181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:25.959068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba2baa-3844-4e69-b195-ad547b984181 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:25.962815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732897f-2fab-486f-aa67-a6c956750a8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:25.963433 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732897f-2fab-486f-aa67-a6c956750a8d service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:25.963723 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732897f-2fab-486f-aa67-a6c956750a8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:25.963997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732897f-2fab-486f-aa67-a6c956750a8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:25.977075 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a732897f-2fab-486f-aa67-a6c956750a8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:25.978616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a732897f-2fab-486f-aa67-a6c956750a8d service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:25.989272 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1220/2441] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:25 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:35:26.078758 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165a775-38c7-4c36-ba4d-816f5b6db668 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:26.089603 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8165a775-38c7-4c36-ba4d-816f5b6db668 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:26.093813 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1eaa15-5622-4efe-a37a-6cfe4bd62da6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:26.094495 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1eaa15-5622-4efe-a37a-6cfe4bd62da6 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:26.094845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1eaa15-5622-4efe-a37a-6cfe4bd62da6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:26.095185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1eaa15-5622-4efe-a37a-6cfe4bd62da6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:26.127645 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:26.159543 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1eaa15-5622-4efe-a37a-6cfe4bd62da6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:26.162146 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1eaa15-5622-4efe-a37a-6cfe4bd62da6 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:26.171512 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1222/2442] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:26 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:35:30.372308 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e169b29-a7e4-4d2b-a361-d6294889525b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:30.372804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e169b29-a7e4-4d2b-a361-d6294889525b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:30.372906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e169b29-a7e4-4d2b-a361-d6294889525b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:30.392209 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1e169b29-a7e4-4d2b-a361-d6294889525b None None] MFA Rules not processed for user `85626350279c40ea99da93b9ed6994b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:30.393385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1e169b29-a7e4-4d2b-a361-d6294889525b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:30.425015 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:30.453363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1221/2443] 10.222.0.39 () {58 vars in 1009 bytes} [Thu Jul 28 22:35:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:30.477859 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1eaa15-5622-4efe-a37a-6cfe4bd62da6 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:30.479361 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2f1eaa15-5622-4efe-a37a-6cfe4bd62da6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:30.548576 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1eaa15-5622-4efe-a37a-6cfe4bd62da6 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:30.553630 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037756e-6dda-4e9b-993f-0891b6f51ccb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9107872f147f6ab8698b67bb545af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:30.554275 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037756e-6dda-4e9b-993f-0891b6f51ccb service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:30.554581 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037756e-6dda-4e9b-993f-0891b6f51ccb service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:30.554855 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037756e-6dda-4e9b-993f-0891b6f51ccb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:30.563282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037756e-6dda-4e9b-993f-0891b6f51ccb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:30.564864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8037756e-6dda-4e9b-993f-0891b6f51ccb service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:30.574111 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1223/2444] 10.222.0.39 () {60 vars in 1384 bytes} [Thu Jul 28 22:35:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 28 22:35:34.512131 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e169b29-a7e4-4d2b-a361-d6294889525b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:34.610445 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e169b29-a7e4-4d2b-a361-d6294889525b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:34.616791 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6c582-3a4e-4648-8c5a-41415cf4439e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:34.618157 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6c582-3a4e-4648-8c5a-41415cf4439e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:34.618806 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6c582-3a4e-4648-8c5a-41415cf4439e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:34.619320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6c582-3a4e-4648-8c5a-41415cf4439e admin admin] PATH_INFO: `/v3/users/ddf962750dde4852a841b5e6cf2bfc14` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:34.621134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6c582-3a4e-4648-8c5a-41415cf4439e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddf962750dde4852a841b5e6cf2bfc14)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:34.624088 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6c582-3a4e-4648-8c5a-41415cf4439e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:34.688846 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-13e6c582-3a4e-4648-8c5a-41415cf4439e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddf962750dde4852a841b5e6cf2bfc14'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:34.699052 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-13e6c582-3a4e-4648-8c5a-41415cf4439e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddf962750dde4852a841b5e6cf2bfc14'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:34.700943 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-13e6c582-3a4e-4648-8c5a-41415cf4439e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddf962750dde4852a841b5e6cf2bfc14'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:34.700943 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-13e6c582-3a4e-4648-8c5a-41415cf4439e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddf962750dde4852a841b5e6cf2bfc14'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:34.718229 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-13e6c582-3a4e-4648-8c5a-41415cf4439e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddf962750dde4852a841b5e6cf2bfc14'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:34.734813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1222/2445] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:35:34 2022] DELETE /identity/v3/users/ddf962750dde4852a841b5e6cf2bfc14 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:34.740428 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a4b02-ab6a-447e-a0bf-2224045a0e87 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:34.741004 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a4b02-ab6a-447e-a0bf-2224045a0e87 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:34.741004 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a4b02-ab6a-447e-a0bf-2224045a0e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:34.762879 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-270a4b02-ab6a-447e-a0bf-2224045a0e87 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:34.764611 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-270a4b02-ab6a-447e-a0bf-2224045a0e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:34.852276 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1224/2446] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:34.865479 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6c582-3a4e-4648-8c5a-41415cf4439e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:34.867335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-13e6c582-3a4e-4648-8c5a-41415cf4439e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:34.971398 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6c582-3a4e-4648-8c5a-41415cf4439e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:34.979001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c839e-8fe1-4f23-8f65-494bc2610e12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:34.980422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c839e-8fe1-4f23-8f65-494bc2610e12 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:34.980422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c839e-8fe1-4f23-8f65-494bc2610e12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:34.980422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c839e-8fe1-4f23-8f65-494bc2610e12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:35.005728 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c839e-8fe1-4f23-8f65-494bc2610e12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:35.007335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c839e-8fe1-4f23-8f65-494bc2610e12 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:35.013077 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:35.041340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1223/2447] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:35:35.383953 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a4b02-ab6a-447e-a0bf-2224045a0e87 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:35.385772 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-270a4b02-ab6a-447e-a0bf-2224045a0e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:35.413696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a4b02-ab6a-447e-a0bf-2224045a0e87 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:35.437610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609d69b-6f48-4ea1-b07d-e5feabf03151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:35.439342 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609d69b-6f48-4ea1-b07d-e5feabf03151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:35.441743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609d69b-6f48-4ea1-b07d-e5feabf03151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:35.442649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609d69b-6f48-4ea1-b07d-e5feabf03151 admin admin] PATH_INFO: `/v3/projects/4acdcdd4c28d417f8e373bd1c7487413` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:35.444728 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609d69b-6f48-4ea1-b07d-e5feabf03151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4acdcdd4c28d417f8e373bd1c7487413)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:35.446823 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609d69b-6f48-4ea1-b07d-e5feabf03151 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:35.542778 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d609d69b-6f48-4ea1-b07d-e5feabf03151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4acdcdd4c28d417f8e373bd1c7487413', 'request_id': 'req-d609d69b-6f48-4ea1-b07d-e5feabf03151'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:35.556743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d609d69b-6f48-4ea1-b07d-e5feabf03151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4acdcdd4c28d417f8e373bd1c7487413', 'request_id': 'req-d609d69b-6f48-4ea1-b07d-e5feabf03151'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:35.560188 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:35.574524 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d609d69b-6f48-4ea1-b07d-e5feabf03151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4acdcdd4c28d417f8e373bd1c7487413', 'request_id': 'req-d609d69b-6f48-4ea1-b07d-e5feabf03151'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:35.590988 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d609d69b-6f48-4ea1-b07d-e5feabf03151 admin admin] The token cache is being invalidate because project 4acdcdd4c28d417f8e373bd1c7487413 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:35:35.591458 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d609d69b-6f48-4ea1-b07d-e5feabf03151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:35.592180 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-d609d69b-6f48-4ea1-b07d-e5feabf03151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:35.593985 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1225/2448] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:35:35 2022] DELETE /identity/v3/projects/4acdcdd4c28d417f8e373bd1c7487413 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:35.598204 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c839e-8fe1-4f23-8f65-494bc2610e12 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:35.608222 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c839e-8fe1-4f23-8f65-494bc2610e12 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:35.693130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba681b32-2041-4c84-b1db-91cd7f72cffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:35.693929 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba681b32-2041-4c84-b1db-91cd7f72cffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:35.694282 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba681b32-2041-4c84-b1db-91cd7f72cffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:35.694621 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba681b32-2041-4c84-b1db-91cd7f72cffe admin admin] PATH_INFO: `/v3/users/acbd5bd3de2d4161aaafe9b05d7bb209` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:35.695866 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba681b32-2041-4c84-b1db-91cd7f72cffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=acbd5bd3de2d4161aaafe9b05d7bb209)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:35.698055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba681b32-2041-4c84-b1db-91cd7f72cffe admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:35.759474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ba681b32-2041-4c84-b1db-91cd7f72cffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acbd5bd3de2d4161aaafe9b05d7bb209'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:35.767823 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ba681b32-2041-4c84-b1db-91cd7f72cffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acbd5bd3de2d4161aaafe9b05d7bb209'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:35.768674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ba681b32-2041-4c84-b1db-91cd7f72cffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acbd5bd3de2d4161aaafe9b05d7bb209'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:35.769331 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ba681b32-2041-4c84-b1db-91cd7f72cffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acbd5bd3de2d4161aaafe9b05d7bb209'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:35.786889 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ba681b32-2041-4c84-b1db-91cd7f72cffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acbd5bd3de2d4161aaafe9b05d7bb209'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:35.809953 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1224/2449] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:35:35 2022] DELETE /identity/v3/users/acbd5bd3de2d4161aaafe9b05d7bb209 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:36.280238 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609d69b-6f48-4ea1-b07d-e5feabf03151 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:36.280238 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d609d69b-6f48-4ea1-b07d-e5feabf03151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:36.303036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609d69b-6f48-4ea1-b07d-e5feabf03151 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:36.385910 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7a189b-c61a-4937-8611-3e157d5abeb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:36.386577 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a189b-c61a-4937-8611-3e157d5abeb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:36.386863 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a189b-c61a-4937-8611-3e157d5abeb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:36.387142 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a189b-c61a-4937-8611-3e157d5abeb0 admin admin] PATH_INFO: `/v3/projects/b71c38d365524f079227e87f0d84e193` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:36.388363 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a189b-c61a-4937-8611-3e157d5abeb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b71c38d365524f079227e87f0d84e193)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:36.390824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7a189b-c61a-4937-8611-3e157d5abeb0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:36.460591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cc7a189b-c61a-4937-8611-3e157d5abeb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b71c38d365524f079227e87f0d84e193', 'request_id': 'req-cc7a189b-c61a-4937-8611-3e157d5abeb0'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:36.471673 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cc7a189b-c61a-4937-8611-3e157d5abeb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b71c38d365524f079227e87f0d84e193', 'request_id': 'req-cc7a189b-c61a-4937-8611-3e157d5abeb0'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:36.494240 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cc7a189b-c61a-4937-8611-3e157d5abeb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b71c38d365524f079227e87f0d84e193', 'request_id': 'req-cc7a189b-c61a-4937-8611-3e157d5abeb0'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:36.512507 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cc7a189b-c61a-4937-8611-3e157d5abeb0 admin admin] The token cache is being invalidate because project b71c38d365524f079227e87f0d84e193 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:35:36.512870 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cc7a189b-c61a-4937-8611-3e157d5abeb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:36.513520 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cc7a189b-c61a-4937-8611-3e157d5abeb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:36.530079 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc1a2d-1334-4577-bfe7-40ae6b7dc06c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:36.530079 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc1a2d-1334-4577-bfe7-40ae6b7dc06c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:36.530079 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc1a2d-1334-4577-bfe7-40ae6b7dc06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:36.531815 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1226/2450] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:35:36 2022] DELETE /identity/v3/projects/b71c38d365524f079227e87f0d84e193 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:36.563534 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-45fc1a2d-1334-4577-bfe7-40ae6b7dc06c None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:36.564845 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-45fc1a2d-1334-4577-bfe7-40ae6b7dc06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:36.720481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1225/2451] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:36.730241 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d931af-496d-4e32-b631-057d5c05929a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:36.730589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d931af-496d-4e32-b631-057d5c05929a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:36.730788 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d931af-496d-4e32-b631-057d5c05929a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:36.775822 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-61d931af-496d-4e32-b631-057d5c05929a None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:36.777167 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-61d931af-496d-4e32-b631-057d5c05929a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:36.790783 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1227/2452] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:36.796545 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc1a2d-1334-4577-bfe7-40ae6b7dc06c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:36.828641 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc1a2d-1334-4577-bfe7-40ae6b7dc06c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:36.834903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc7948-1481-4926-979e-d7d411e68e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:36.835533 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc7948-1481-4926-979e-d7d411e68e47 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:36.835786 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc7948-1481-4926-979e-d7d411e68e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:36.836385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbc7948-1481-4926-979e-d7d411e68e47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:36.836916 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc7948-1481-4926-979e-d7d411e68e47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:35:36.837254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc7948-1481-4926-979e-d7d411e68e47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:36.839056 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbc7948-1481-4926-979e-d7d411e68e47 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:36.857900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1226/2453] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:35:36.858707 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d931af-496d-4e32-b631-057d5c05929a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:36.868428 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d931af-496d-4e32-b631-057d5c05929a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:36.875746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2099b2c-384f-4f12-9678-e89725ea3db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:36.877940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2099b2c-384f-4f12-9678-e89725ea3db1 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:36.878391 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2099b2c-384f-4f12-9678-e89725ea3db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:36.878796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2099b2c-384f-4f12-9678-e89725ea3db1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:36.879804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2099b2c-384f-4f12-9678-e89725ea3db1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:36.882988 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2099b2c-384f-4f12-9678-e89725ea3db1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:36.949387 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1228/2454] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:35:36 2022] POST /identity/v3/projects => generated 383 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:36.953594 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc7948-1481-4926-979e-d7d411e68e47 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:36.966004 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbc7948-1481-4926-979e-d7d411e68e47 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:36.970940 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:37.060582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad13a7-281a-431d-9a68-664df206982f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:37.062242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad13a7-281a-431d-9a68-664df206982f admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:37.062914 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad13a7-281a-431d-9a68-664df206982f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:37.063291 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ad13a7-281a-431d-9a68-664df206982f admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:37.064073 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ad13a7-281a-431d-9a68-664df206982f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:37.067162 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ad13a7-281a-431d-9a68-664df206982f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:37.101409 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1227/2455] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:35:36 2022] POST /identity/v3/users => generated 292 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:37.102911 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2099b2c-384f-4f12-9678-e89725ea3db1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:37.118355 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2099b2c-384f-4f12-9678-e89725ea3db1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:37.131174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33be6c-f85e-4aa6-aee9-7b19ca228934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:37.131970 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33be6c-f85e-4aa6-aee9-7b19ca228934 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:37.132356 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33be6c-f85e-4aa6-aee9-7b19ca228934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:37.132722 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33be6c-f85e-4aa6-aee9-7b19ca228934 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:37.133549 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa33be6c-f85e-4aa6-aee9-7b19ca228934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:37.137473 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa33be6c-f85e-4aa6-aee9-7b19ca228934 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:37.158090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1229/2456] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:37.164596 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad13a7-281a-431d-9a68-664df206982f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:37.181844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ad13a7-281a-431d-9a68-664df206982f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:37.187951 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1afa72-62cf-483d-b1f6-dfd9649a014e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:37.188722 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1afa72-62cf-483d-b1f6-dfd9649a014e admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:37.189099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1afa72-62cf-483d-b1f6-dfd9649a014e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:37.189475 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1afa72-62cf-483d-b1f6-dfd9649a014e admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:37.190311 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1afa72-62cf-483d-b1f6-dfd9649a014e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:37.192433 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1afa72-62cf-483d-b1f6-dfd9649a014e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:37.207698 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1228/2457] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:35:37.212083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33be6c-f85e-4aa6-aee9-7b19ca228934 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:37.225322 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33be6c-f85e-4aa6-aee9-7b19ca228934 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:37.230443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8081468-c724-423d-bae2-50ce640e14cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:37.231237 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8081468-c724-423d-bae2-50ce640e14cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:37.231622 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8081468-c724-423d-bae2-50ce640e14cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:37.232005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8081468-c724-423d-bae2-50ce640e14cd admin admin] PATH_INFO: `/v3/projects/6dc16dd2e2664546af3513bf12c4ad2e/users/81abbf92898c488b82ec2fb2808a8e4c/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:37.251402 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8081468-c724-423d-bae2-50ce640e14cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dc16dd2e2664546af3513bf12c4ad2e, user_id=81abbf92898c488b82ec2fb2808a8e4c, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:37.255333 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8081468-c724-423d-bae2-50ce640e14cd admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:37.281169 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1230/2458] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:35:37 2022] PUT /identity/v3/projects/6dc16dd2e2664546af3513bf12c4ad2e/users/81abbf92898c488b82ec2fb2808a8e4c/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:37.292236 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3c321-f53d-42aa-9d62-4bb5211c6241 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:37.292705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3c321-f53d-42aa-9d62-4bb5211c6241 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:37.293139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed3c321-f53d-42aa-9d62-4bb5211c6241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:37.345720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-fed3c321-f53d-42aa-9d62-4bb5211c6241 None None] MFA Rules not processed for user `81abbf92898c488b82ec2fb2808a8e4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:37.348138 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fed3c321-f53d-42aa-9d62-4bb5211c6241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:37.490948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1229/2459] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:37.495535 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8081468-c724-423d-bae2-50ce640e14cd admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:37.522995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:37.606167 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8081468-c724-423d-bae2-50ce640e14cd admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:37.612776 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665e5ce-7df5-43d2-a304-b954272cdf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:37.614741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665e5ce-7df5-43d2-a304-b954272cdf46 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:37.615356 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665e5ce-7df5-43d2-a304-b954272cdf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:37.615894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6665e5ce-7df5-43d2-a304-b954272cdf46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:37.616704 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665e5ce-7df5-43d2-a304-b954272cdf46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:37.624399 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6665e5ce-7df5-43d2-a304-b954272cdf46 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:37.681297 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1231/2460] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:35:37 2022] POST /identity/v3/projects => generated 383 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:37.682490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3c321-f53d-42aa-9d62-4bb5211c6241 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:37.699673 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed3c321-f53d-42aa-9d62-4bb5211c6241 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:37.832700 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7910b07c-418b-419b-b684-b4f8b0f5bdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:37.833382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910b07c-418b-419b-b684-b4f8b0f5bdd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:37.833768 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910b07c-418b-419b-b684-b4f8b0f5bdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:37.834146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910b07c-418b-419b-b684-b4f8b0f5bdd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:37.834936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7910b07c-418b-419b-b684-b4f8b0f5bdd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:37.837096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7910b07c-418b-419b-b684-b4f8b0f5bdd3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:37.869259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1230/2461] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:35:37 2022] POST /identity/v3/users => generated 298 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:37.873079 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665e5ce-7df5-43d2-a304-b954272cdf46 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:37.888346 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6665e5ce-7df5-43d2-a304-b954272cdf46 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:37.904110 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71584ed7-63a4-441f-9f69-a48df2c30663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:37.904763 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71584ed7-63a4-441f-9f69-a48df2c30663 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:37.905047 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71584ed7-63a4-441f-9f69-a48df2c30663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:37.905328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71584ed7-63a4-441f-9f69-a48df2c30663 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:37.906259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71584ed7-63a4-441f-9f69-a48df2c30663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:37.907904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71584ed7-63a4-441f-9f69-a48df2c30663 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:37.950944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1232/2462] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:37.952226 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7910b07c-418b-419b-b684-b4f8b0f5bdd3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:37.959833 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:37.965410 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7910b07c-418b-419b-b684-b4f8b0f5bdd3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:37.978232 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f030f6-222c-4b78-a36f-f9e94850299d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:37.978924 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f030f6-222c-4b78-a36f-f9e94850299d admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:37.979212 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f030f6-222c-4b78-a36f-f9e94850299d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:37.979494 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f030f6-222c-4b78-a36f-f9e94850299d admin admin] PATH_INFO: `/v3/projects/1ae912bf26144a66858cee7f95c94526/users/f36cd44c28874cee8c87d3e5470c26f4/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:38.055665 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f030f6-222c-4b78-a36f-f9e94850299d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae912bf26144a66858cee7f95c94526, user_id=f36cd44c28874cee8c87d3e5470c26f4, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:38.058450 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f030f6-222c-4b78-a36f-f9e94850299d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:38.124915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71584ed7-63a4-441f-9f69-a48df2c30663 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:38.143050 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1231/2463] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:35:37 2022] PUT /identity/v3/projects/1ae912bf26144a66858cee7f95c94526/users/f36cd44c28874cee8c87d3e5470c26f4/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:38.147133 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71584ed7-63a4-441f-9f69-a48df2c30663 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:38.284917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b3537-0d3b-44b7-b71c-a21bfe2d3f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:38.285742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b3537-0d3b-44b7-b71c-a21bfe2d3f87 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:38.285956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b3537-0d3b-44b7-b71c-a21bfe2d3f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:38.286203 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b3537-0d3b-44b7-b71c-a21bfe2d3f87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:38.286903 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b3537-0d3b-44b7-b71c-a21bfe2d3f87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:38.288541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b3537-0d3b-44b7-b71c-a21bfe2d3f87 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:38.313180 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f030f6-222c-4b78-a36f-f9e94850299d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:38.313511 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1233/2464] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:38.332736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f030f6-222c-4b78-a36f-f9e94850299d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:38.343310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c927e6-86b5-46f2-b80d-d4727a96476d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:38.344068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c927e6-86b5-46f2-b80d-d4727a96476d admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:38.344354 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c927e6-86b5-46f2-b80d-d4727a96476d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:38.344656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c927e6-86b5-46f2-b80d-d4727a96476d admin admin] PATH_INFO: `/v3/projects/1ae912bf26144a66858cee7f95c94526/users/f36cd44c28874cee8c87d3e5470c26f4/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:38.363937 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c927e6-86b5-46f2-b80d-d4727a96476d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ae912bf26144a66858cee7f95c94526, user_id=f36cd44c28874cee8c87d3e5470c26f4, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:38.370367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c927e6-86b5-46f2-b80d-d4727a96476d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:38.386296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1232/2465] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:35:38 2022] PUT /identity/v3/projects/1ae912bf26144a66858cee7f95c94526/users/f36cd44c28874cee8c87d3e5470c26f4/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:35:38.398682 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d4c5e-0fc7-444b-8fac-bee391d7d002 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:38.399209 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d4c5e-0fc7-444b-8fac-bee391d7d002 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:38.399659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d4c5e-0fc7-444b-8fac-bee391d7d002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:38.440627 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-201d4c5e-0fc7-444b-8fac-bee391d7d002 None None] MFA Rules not processed for user `f36cd44c28874cee8c87d3e5470c26f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:38.442043 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-201d4c5e-0fc7-444b-8fac-bee391d7d002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:38.642565 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1234/2466] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:38.658120 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c927e6-86b5-46f2-b80d-d4727a96476d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:38.663407 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-82c927e6-86b5-46f2-b80d-d4727a96476d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:38.763617 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c927e6-86b5-46f2-b80d-d4727a96476d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:38.777004 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45a3f4-c9ef-4e6f-82b9-1e09ccc4da6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:38.777759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45a3f4-c9ef-4e6f-82b9-1e09ccc4da6f service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:38.778075 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45a3f4-c9ef-4e6f-82b9-1e09ccc4da6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:38.778348 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb45a3f4-c9ef-4e6f-82b9-1e09ccc4da6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:38.804884 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb45a3f4-c9ef-4e6f-82b9-1e09ccc4da6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:38.810824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb45a3f4-c9ef-4e6f-82b9-1e09ccc4da6f service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:38.833889 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1233/2467] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:35:38.961938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d4c5e-0fc7-444b-8fac-bee391d7d002 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:38.979998 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d4c5e-0fc7-444b-8fac-bee391d7d002 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:38.985160 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163d3bd8-0eb3-4f26-9ad9-1a772db00725 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:38.985821 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d3bd8-0eb3-4f26-9ad9-1a772db00725 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:38.987332 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d3bd8-0eb3-4f26-9ad9-1a772db00725 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:38.988214 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163d3bd8-0eb3-4f26-9ad9-1a772db00725 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:39.002366 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:39.111255 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d3bd8-0eb3-4f26-9ad9-1a772db00725 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:39.114109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163d3bd8-0eb3-4f26-9ad9-1a772db00725 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:39.136647 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1235/2468] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:35:39.722950 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45a3f4-c9ef-4e6f-82b9-1e09ccc4da6f service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:39.914953 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb45a3f4-c9ef-4e6f-82b9-1e09ccc4da6f service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:39.928383 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec610a-5d4c-4407-ba46-e0c565ffc764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:39.929163 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec610a-5d4c-4407-ba46-e0c565ffc764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:39.929730 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec610a-5d4c-4407-ba46-e0c565ffc764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:39.930231 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec610a-5d4c-4407-ba46-e0c565ffc764 admin admin] PATH_INFO: `/v3/users/81abbf92898c488b82ec2fb2808a8e4c` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:39.931661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec610a-5d4c-4407-ba46-e0c565ffc764 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81abbf92898c488b82ec2fb2808a8e4c)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:39.933873 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec610a-5d4c-4407-ba46-e0c565ffc764 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:40.015714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f5ec610a-5d4c-4407-ba46-e0c565ffc764 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81abbf92898c488b82ec2fb2808a8e4c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:40.026545 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f5ec610a-5d4c-4407-ba46-e0c565ffc764 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81abbf92898c488b82ec2fb2808a8e4c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:40.027514 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f5ec610a-5d4c-4407-ba46-e0c565ffc764 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81abbf92898c488b82ec2fb2808a8e4c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:40.028605 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f5ec610a-5d4c-4407-ba46-e0c565ffc764 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81abbf92898c488b82ec2fb2808a8e4c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:40.045513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:40.047050 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f5ec610a-5d4c-4407-ba46-e0c565ffc764 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81abbf92898c488b82ec2fb2808a8e4c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:40.084209 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1234/2469] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:35:39 2022] DELETE /identity/v3/users/81abbf92898c488b82ec2fb2808a8e4c => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:40.088814 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432b990-d8fc-4c54-b5e7-89e2f1f83ac9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:40.089339 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432b990-d8fc-4c54-b5e7-89e2f1f83ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:40.089634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d432b990-d8fc-4c54-b5e7-89e2f1f83ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:40.141395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d432b990-d8fc-4c54-b5e7-89e2f1f83ac9 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:40.142774 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d432b990-d8fc-4c54-b5e7-89e2f1f83ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:40.301595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1236/2470] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:40.322769 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec610a-5d4c-4407-ba46-e0c565ffc764 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:40.324304 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f5ec610a-5d4c-4407-ba46-e0c565ffc764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:40.469111 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec610a-5d4c-4407-ba46-e0c565ffc764 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:40.490079 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba8550-d0e7-42b4-877c-356f040aba4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:40.490738 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba8550-d0e7-42b4-877c-356f040aba4f service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:40.491031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba8550-d0e7-42b4-877c-356f040aba4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:40.491404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba8550-d0e7-42b4-877c-356f040aba4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:40.513408 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba8550-d0e7-42b4-877c-356f040aba4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:40.514928 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ba8550-d0e7-42b4-877c-356f040aba4f service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:40.541710 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1235/2471] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:35:41.151117 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432b990-d8fc-4c54-b5e7-89e2f1f83ac9 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:41.154141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d432b990-d8fc-4c54-b5e7-89e2f1f83ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:41.194726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d432b990-d8fc-4c54-b5e7-89e2f1f83ac9 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:41.205419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179da50-996e-43d6-94e3-dbadefe36a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:41.208432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179da50-996e-43d6-94e3-dbadefe36a67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:41.208776 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179da50-996e-43d6-94e3-dbadefe36a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:41.209109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179da50-996e-43d6-94e3-dbadefe36a67 admin admin] PATH_INFO: `/v3/projects/6dc16dd2e2664546af3513bf12c4ad2e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:41.211645 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e179da50-996e-43d6-94e3-dbadefe36a67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dc16dd2e2664546af3513bf12c4ad2e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:41.213840 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e179da50-996e-43d6-94e3-dbadefe36a67 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:41.301070 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e179da50-996e-43d6-94e3-dbadefe36a67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dc16dd2e2664546af3513bf12c4ad2e', 'request_id': 'req-e179da50-996e-43d6-94e3-dbadefe36a67'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:41.332948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e179da50-996e-43d6-94e3-dbadefe36a67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dc16dd2e2664546af3513bf12c4ad2e', 'request_id': 'req-e179da50-996e-43d6-94e3-dbadefe36a67'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:41.340218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e179da50-996e-43d6-94e3-dbadefe36a67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dc16dd2e2664546af3513bf12c4ad2e', 'request_id': 'req-e179da50-996e-43d6-94e3-dbadefe36a67'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:41.356674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e179da50-996e-43d6-94e3-dbadefe36a67 admin admin] The token cache is being invalidate because project 6dc16dd2e2664546af3513bf12c4ad2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:35:41.357417 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e179da50-996e-43d6-94e3-dbadefe36a67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:41.358421 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e179da50-996e-43d6-94e3-dbadefe36a67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:41.363717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1237/2472] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:35:41 2022] DELETE /identity/v3/projects/6dc16dd2e2664546af3513bf12c4ad2e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:41.364613 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba8550-d0e7-42b4-877c-356f040aba4f service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:41.385322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ba8550-d0e7-42b4-877c-356f040aba4f service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:41.492364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b19d88-54bb-41a1-bbf0-fdc109fac75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:41.493663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b19d88-54bb-41a1-bbf0-fdc109fac75b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:41.494092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b19d88-54bb-41a1-bbf0-fdc109fac75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:41.494354 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b19d88-54bb-41a1-bbf0-fdc109fac75b admin admin] PATH_INFO: `/v3/users/f36cd44c28874cee8c87d3e5470c26f4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:41.495640 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b19d88-54bb-41a1-bbf0-fdc109fac75b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f36cd44c28874cee8c87d3e5470c26f4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:41.497646 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b19d88-54bb-41a1-bbf0-fdc109fac75b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:41.581589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c3b19d88-54bb-41a1-bbf0-fdc109fac75b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f36cd44c28874cee8c87d3e5470c26f4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:41.590285 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c3b19d88-54bb-41a1-bbf0-fdc109fac75b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f36cd44c28874cee8c87d3e5470c26f4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:41.591042 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c3b19d88-54bb-41a1-bbf0-fdc109fac75b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f36cd44c28874cee8c87d3e5470c26f4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:41.591652 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c3b19d88-54bb-41a1-bbf0-fdc109fac75b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f36cd44c28874cee8c87d3e5470c26f4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:41.608529 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c3b19d88-54bb-41a1-bbf0-fdc109fac75b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f36cd44c28874cee8c87d3e5470c26f4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:41.639450 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1236/2473] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:35:41 2022] DELETE /identity/v3/users/f36cd44c28874cee8c87d3e5470c26f4 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:42.033812 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179da50-996e-43d6-94e3-dbadefe36a67 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:42.037804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e179da50-996e-43d6-94e3-dbadefe36a67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:42.060420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179da50-996e-43d6-94e3-dbadefe36a67 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:42.140198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:42.215348 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c80d02-938a-4f0f-bd04-cb6080b879e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:42.216012 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c80d02-938a-4f0f-bd04-cb6080b879e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:42.216366 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c80d02-938a-4f0f-bd04-cb6080b879e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:42.216651 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c80d02-938a-4f0f-bd04-cb6080b879e6 admin admin] PATH_INFO: `/v3/projects/1ae912bf26144a66858cee7f95c94526` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:42.217998 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c80d02-938a-4f0f-bd04-cb6080b879e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ae912bf26144a66858cee7f95c94526)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:42.220085 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c80d02-938a-4f0f-bd04-cb6080b879e6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:42.306204 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-66c80d02-938a-4f0f-bd04-cb6080b879e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ae912bf26144a66858cee7f95c94526', 'request_id': 'req-66c80d02-938a-4f0f-bd04-cb6080b879e6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:42.316732 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-66c80d02-938a-4f0f-bd04-cb6080b879e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ae912bf26144a66858cee7f95c94526', 'request_id': 'req-66c80d02-938a-4f0f-bd04-cb6080b879e6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:42.324356 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-66c80d02-938a-4f0f-bd04-cb6080b879e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ae912bf26144a66858cee7f95c94526', 'request_id': 'req-66c80d02-938a-4f0f-bd04-cb6080b879e6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:42.341217 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-66c80d02-938a-4f0f-bd04-cb6080b879e6 admin admin] The token cache is being invalidate because project 1ae912bf26144a66858cee7f95c94526 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:35:42.341924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-66c80d02-938a-4f0f-bd04-cb6080b879e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:42.344487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-66c80d02-938a-4f0f-bd04-cb6080b879e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:42.347923 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1238/2474] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:35:42 2022] DELETE /identity/v3/projects/1ae912bf26144a66858cee7f95c94526 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:42.352998 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b5be0-b48a-4aa0-a47c-b7b313769b1f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:42.353291 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b5be0-b48a-4aa0-a47c-b7b313769b1f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:42.353593 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b5be0-b48a-4aa0-a47c-b7b313769b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:42.383337 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8e4b5be0-b48a-4aa0-a47c-b7b313769b1f None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:42.384797 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8e4b5be0-b48a-4aa0-a47c-b7b313769b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:42.390903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:42.478564 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1237/2475] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:42.480222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8b05c-2a30-452a-943b-aad88e7c2a5a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:42.480660 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8b05c-2a30-452a-943b-aad88e7c2a5a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:42.481173 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8b05c-2a30-452a-943b-aad88e7c2a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:42.543696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f7e8b05c-2a30-452a-943b-aad88e7c2a5a None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:42.553800 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f7e8b05c-2a30-452a-943b-aad88e7c2a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:42.594265 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1239/2476] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:35:42.596180 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b5be0-b48a-4aa0-a47c-b7b313769b1f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:42.629953 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b5be0-b48a-4aa0-a47c-b7b313769b1f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:42.646625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98c2c8-7de4-4f62-a14e-0423701738d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:42.649844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98c2c8-7de4-4f62-a14e-0423701738d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:42.650151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98c2c8-7de4-4f62-a14e-0423701738d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:42.651514 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98c2c8-7de4-4f62-a14e-0423701738d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:42.653203 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98c2c8-7de4-4f62-a14e-0423701738d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:35:42.653956 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98c2c8-7de4-4f62-a14e-0423701738d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:42.658379 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d98c2c8-7de4-4f62-a14e-0423701738d8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:42.705563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1238/2477] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:35:42.709366 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8b05c-2a30-452a-943b-aad88e7c2a5a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:42.720873 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8b05c-2a30-452a-943b-aad88e7c2a5a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:42.736981 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b4b79-dac6-4594-bc18-03b0fa48e0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:42.742939 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b4b79-dac6-4594-bc18-03b0fa48e0ab admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:42.743227 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b4b79-dac6-4594-bc18-03b0fa48e0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:42.743500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6b4b79-dac6-4594-bc18-03b0fa48e0ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:42.744209 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b4b79-dac6-4594-bc18-03b0fa48e0ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:42.749582 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6b4b79-dac6-4594-bc18-03b0fa48e0ab admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:42.813783 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1240/2478] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:35:42 2022] POST /identity/v3/projects => generated 371 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:42.818482 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98c2c8-7de4-4f62-a14e-0423701738d8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:42.833344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98c2c8-7de4-4f62-a14e-0423701738d8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:42.919541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f0c19-2b08-4662-8bf8-dfe95264b3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:42.921281 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f0c19-2b08-4662-8bf8-dfe95264b3bd admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:42.925575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f0c19-2b08-4662-8bf8-dfe95264b3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:42.926254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f0c19-2b08-4662-8bf8-dfe95264b3bd admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:42.927206 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f0c19-2b08-4662-8bf8-dfe95264b3bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:42.929456 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f0c19-2b08-4662-8bf8-dfe95264b3bd admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:42.975493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1239/2479] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:35:42 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:42.979617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b4b79-dac6-4594-bc18-03b0fa48e0ab admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:42.995962 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6b4b79-dac6-4594-bc18-03b0fa48e0ab admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:43.007882 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201f68d-517c-49eb-a352-b813b9a5bc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:43.008635 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201f68d-517c-49eb-a352-b813b9a5bc93 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:43.009045 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201f68d-517c-49eb-a352-b813b9a5bc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:43.009398 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201f68d-517c-49eb-a352-b813b9a5bc93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:43.010449 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201f68d-517c-49eb-a352-b813b9a5bc93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:43.012649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201f68d-517c-49eb-a352-b813b9a5bc93 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:43.028764 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1241/2480] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:43.030249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f0c19-2b08-4662-8bf8-dfe95264b3bd admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:43.044479 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f0c19-2b08-4662-8bf8-dfe95264b3bd admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:43.056449 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e29d75-bedd-4769-b054-435c4212c12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:43.057610 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e29d75-bedd-4769-b054-435c4212c12a admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:43.058853 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e29d75-bedd-4769-b054-435c4212c12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:43.059263 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e29d75-bedd-4769-b054-435c4212c12a admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:43.060338 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e29d75-bedd-4769-b054-435c4212c12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:43.062927 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e29d75-bedd-4769-b054-435c4212c12a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:43.081969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1240/2481] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:43.086962 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201f68d-517c-49eb-a352-b813b9a5bc93 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:43.100357 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201f68d-517c-49eb-a352-b813b9a5bc93 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:43.108423 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90dbe8-7dc9-4117-964d-a5b2fbdbdce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:43.109464 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90dbe8-7dc9-4117-964d-a5b2fbdbdce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:43.110172 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90dbe8-7dc9-4117-964d-a5b2fbdbdce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:43.110511 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90dbe8-7dc9-4117-964d-a5b2fbdbdce3 admin admin] PATH_INFO: `/v3/projects/a4f75425fc254368bd7870a27d8b0ebe/users/21faee3ebc194e709a933b6fbb7640d4/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:43.140977 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90dbe8-7dc9-4117-964d-a5b2fbdbdce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4f75425fc254368bd7870a27d8b0ebe, user_id=21faee3ebc194e709a933b6fbb7640d4, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:43.147490 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90dbe8-7dc9-4117-964d-a5b2fbdbdce3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:43.172487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1242/2482] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:35:43 2022] PUT /identity/v3/projects/a4f75425fc254368bd7870a27d8b0ebe/users/21faee3ebc194e709a933b6fbb7640d4/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:43.178307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150a4e7-b0a6-42c1-850e-273a3bc5e50c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:43.178845 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150a4e7-b0a6-42c1-850e-273a3bc5e50c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:43.179344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c150a4e7-b0a6-42c1-850e-273a3bc5e50c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:43.203363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c150a4e7-b0a6-42c1-850e-273a3bc5e50c None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:43.204843 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c150a4e7-b0a6-42c1-850e-273a3bc5e50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:43.290555 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1241/2483] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:43.297305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90dbe8-7dc9-4117-964d-a5b2fbdbdce3 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:43.300939 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ee90dbe8-7dc9-4117-964d-a5b2fbdbdce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:43.427341 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90dbe8-7dc9-4117-964d-a5b2fbdbdce3 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:43.440658 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da5dcf-877b-40a6-b2b5-94d45ce0bf76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:43.441516 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da5dcf-877b-40a6-b2b5-94d45ce0bf76 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:43.441936 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da5dcf-877b-40a6-b2b5-94d45ce0bf76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:43.442386 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da5dcf-877b-40a6-b2b5-94d45ce0bf76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:43.455495 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da5dcf-877b-40a6-b2b5-94d45ce0bf76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:43.457597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da5dcf-877b-40a6-b2b5-94d45ce0bf76 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:43.462758 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:43.472815 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1243/2484] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:35:43.795301 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495e860-eeb6-4374-adfe-d3eedac5d85b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:43.795788 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495e860-eeb6-4374-adfe-d3eedac5d85b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:43.795966 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495e860-eeb6-4374-adfe-d3eedac5d85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:43.828683 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5495e860-eeb6-4374-adfe-d3eedac5d85b None None] MFA Rules not processed for user `32eadc02f9224eb28303aab25506ef2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:43.831732 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5495e860-eeb6-4374-adfe-d3eedac5d85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:43.864434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:44.029645 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1242/2485] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:44.114866 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da5dcf-877b-40a6-b2b5-94d45ce0bf76 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:44.139476 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da5dcf-877b-40a6-b2b5-94d45ce0bf76 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:44.149861 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee5220d-387b-4b1c-baaf-df9a500b7ec3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:44.151513 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5220d-387b-4b1c-baaf-df9a500b7ec3 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:44.153437 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5220d-387b-4b1c-baaf-df9a500b7ec3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:44.153953 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee5220d-387b-4b1c-baaf-df9a500b7ec3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:44.161556 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4ee5220d-387b-4b1c-baaf-df9a500b7ec3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:44.200283 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5220d-387b-4b1c-baaf-df9a500b7ec3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:44.202154 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee5220d-387b-4b1c-baaf-df9a500b7ec3 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:44.234879 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1244/2486] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:35:44.903807 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495e860-eeb6-4374-adfe-d3eedac5d85b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:44.907322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5495e860-eeb6-4374-adfe-d3eedac5d85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:44.936137 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495e860-eeb6-4374-adfe-d3eedac5d85b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:44.943916 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843e01e-3bf0-4896-9057-15c3772a7831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:44.944984 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843e01e-3bf0-4896-9057-15c3772a7831 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:44.945481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843e01e-3bf0-4896-9057-15c3772a7831 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:44.946161 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843e01e-3bf0-4896-9057-15c3772a7831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:44.948569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5843e01e-3bf0-4896-9057-15c3772a7831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:44.963904 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5843e01e-3bf0-4896-9057-15c3772a7831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:44.965800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5843e01e-3bf0-4896-9057-15c3772a7831 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:45.017942 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1243/2487] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:35:45.590836 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76904b2-a84a-4d09-afed-96bf7a10bcb6 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:45.591825 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76904b2-a84a-4d09-afed-96bf7a10bcb6 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:45.592341 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76904b2-a84a-4d09-afed-96bf7a10bcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:45.628903 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d76904b2-a84a-4d09-afed-96bf7a10bcb6 None None] MFA Rules not processed for user `21faee3ebc194e709a933b6fbb7640d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:45.630863 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d76904b2-a84a-4d09-afed-96bf7a10bcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:45.746112 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843e01e-3bf0-4896-9057-15c3772a7831 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:45.754432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1245/2488] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:45.758694 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5843e01e-3bf0-4896-9057-15c3772a7831 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:45.763938 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117dc57-5cfb-43d9-a613-5945fa4419c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:45.765598 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117dc57-5cfb-43d9-a613-5945fa4419c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:45.765984 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117dc57-5cfb-43d9-a613-5945fa4419c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:45.766476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117dc57-5cfb-43d9-a613-5945fa4419c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:45.770948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117dc57-5cfb-43d9-a613-5945fa4419c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:45.773190 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117dc57-5cfb-43d9-a613-5945fa4419c0 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:45.837405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1244/2489] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:35:45 2022] POST /identity/v3/projects => generated 371 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:45.839526 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76904b2-a84a-4d09-afed-96bf7a10bcb6 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:45.867398 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76904b2-a84a-4d09-afed-96bf7a10bcb6 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:45.913152 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117dc57-5cfb-43d9-a613-5945fa4419c0 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:46.067451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b13d3-ae5a-4615-9beb-dc97955a38a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:46.068002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b13d3-ae5a-4615-9beb-dc97955a38a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:46.077870 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b13d3-ae5a-4615-9beb-dc97955a38a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:46.078361 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b13d3-ae5a-4615-9beb-dc97955a38a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:46.079157 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b13d3-ae5a-4615-9beb-dc97955a38a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:46.081348 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b13d3-ae5a-4615-9beb-dc97955a38a9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:46.083994 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117dc57-5cfb-43d9-a613-5945fa4419c0 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:46.122225 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9beb094-99c9-4cef-b314-124c1f81ca38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:46.126175 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9beb094-99c9-4cef-b314-124c1f81ca38 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:46.126915 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9beb094-99c9-4cef-b314-124c1f81ca38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:46.127498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9beb094-99c9-4cef-b314-124c1f81ca38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:46.133392 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1246/2490] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:35:45 2022] POST /identity/v3/users => generated 292 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:46.135077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b13d3-ae5a-4615-9beb-dc97955a38a9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:46.142255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f9beb094-99c9-4cef-b314-124c1f81ca38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:46.209672 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b13d3-ae5a-4615-9beb-dc97955a38a9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:46.219365 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61c47a-a82c-4c76-b1f2-b6d4d58ecd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:46.220013 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61c47a-a82c-4c76-b1f2-b6d4d58ecd34 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:46.220327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61c47a-a82c-4c76-b1f2-b6d4d58ecd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:46.220569 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61c47a-a82c-4c76-b1f2-b6d4d58ecd34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:46.221277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61c47a-a82c-4c76-b1f2-b6d4d58ecd34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:46.223753 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61c47a-a82c-4c76-b1f2-b6d4d58ecd34 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:46.258453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1247/2491] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:46.262231 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61c47a-a82c-4c76-b1f2-b6d4d58ecd34 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:46.288292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61c47a-a82c-4c76-b1f2-b6d4d58ecd34 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:46.303099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e875a3-e21c-47b1-912c-6c32c6510592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:46.304090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e875a3-e21c-47b1-912c-6c32c6510592 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:46.304531 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e875a3-e21c-47b1-912c-6c32c6510592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:46.304971 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e875a3-e21c-47b1-912c-6c32c6510592 admin admin] PATH_INFO: `/v3/projects/04fc9eb053f64757a168c04252c7f44f/users/91b3a982f4484b6c9e8ef46494f4c92a/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:46.313126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9beb094-99c9-4cef-b314-124c1f81ca38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:46.314748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9beb094-99c9-4cef-b314-124c1f81ca38 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:46.342251 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e875a3-e21c-47b1-912c-6c32c6510592 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04fc9eb053f64757a168c04252c7f44f, user_id=91b3a982f4484b6c9e8ef46494f4c92a, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:46.344483 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1245/2492] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:35:46.346839 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e875a3-e21c-47b1-912c-6c32c6510592 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:46.381490 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1248/2493] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:35:46 2022] PUT /identity/v3/projects/04fc9eb053f64757a168c04252c7f44f/users/91b3a982f4484b6c9e8ef46494f4c92a/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:46.385157 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9beb094-99c9-4cef-b314-124c1f81ca38 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:46.410565 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9beb094-99c9-4cef-b314-124c1f81ca38 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:46.506997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:46.540095 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d05902c-12d2-44fb-92d9-a828f0510284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:46.540780 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05902c-12d2-44fb-92d9-a828f0510284 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:46.541123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05902c-12d2-44fb-92d9-a828f0510284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:46.541402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d05902c-12d2-44fb-92d9-a828f0510284 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:46.542542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05902c-12d2-44fb-92d9-a828f0510284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:46.544403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d05902c-12d2-44fb-92d9-a828f0510284 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:46.571086 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1246/2494] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:46 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:46.576029 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e875a3-e21c-47b1-912c-6c32c6510592 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:46.595427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e875a3-e21c-47b1-912c-6c32c6510592 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:46.611731 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e54c2-df0e-4fcd-8a6a-969211990471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:46.612588 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e54c2-df0e-4fcd-8a6a-969211990471 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:46.613228 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e54c2-df0e-4fcd-8a6a-969211990471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:46.614323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e54c2-df0e-4fcd-8a6a-969211990471 admin admin] PATH_INFO: `/v3/projects/04fc9eb053f64757a168c04252c7f44f/users/91b3a982f4484b6c9e8ef46494f4c92a/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:46.621871 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e54c2-df0e-4fcd-8a6a-969211990471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04fc9eb053f64757a168c04252c7f44f, user_id=91b3a982f4484b6c9e8ef46494f4c92a, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:46.629132 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e54c2-df0e-4fcd-8a6a-969211990471 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:46.660199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1249/2495] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:35:46 2022] PUT /identity/v3/projects/04fc9eb053f64757a168c04252c7f44f/users/91b3a982f4484b6c9e8ef46494f4c92a/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:51.370245 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b22956-fd3d-41aa-8619-49ef43a73ec3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:51.371852 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b22956-fd3d-41aa-8619-49ef43a73ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:51.371852 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b22956-fd3d-41aa-8619-49ef43a73ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:51.430462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-42b22956-fd3d-41aa-8619-49ef43a73ec3 None None] MFA Rules not processed for user `91b3a982f4484b6c9e8ef46494f4c92a`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:51.431876 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-42b22956-fd3d-41aa-8619-49ef43a73ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:51.623312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1247/2496] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:51 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:51.638143 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e54c2-df0e-4fcd-8a6a-969211990471 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:51.640953 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f96e54c2-df0e-4fcd-8a6a-969211990471 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:51.672903 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:51.743846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e54c2-df0e-4fcd-8a6a-969211990471 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:51.748372 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359f0f9-19cb-4ae0-989f-706c8055c020 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:51.749406 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359f0f9-19cb-4ae0-989f-706c8055c020 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:51.749964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359f0f9-19cb-4ae0-989f-706c8055c020 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:51.750498 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359f0f9-19cb-4ae0-989f-706c8055c020 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:51.881309 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359f0f9-19cb-4ae0-989f-706c8055c020 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:51.883088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359f0f9-19cb-4ae0-989f-706c8055c020 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:51.913679 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1250/2497] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:35:52.202684 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b22956-fd3d-41aa-8619-49ef43a73ec3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:52.204186 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-42b22956-fd3d-41aa-8619-49ef43a73ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:52.406889 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b22956-fd3d-41aa-8619-49ef43a73ec3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:52.422867 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e81b3f-f1da-4ac9-a0a2-9ea4392cea67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:52.426792 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e81b3f-f1da-4ac9-a0a2-9ea4392cea67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:52.427500 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e81b3f-f1da-4ac9-a0a2-9ea4392cea67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:52.427999 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e81b3f-f1da-4ac9-a0a2-9ea4392cea67 admin admin] PATH_INFO: `/v3/users/d38b517910894f7fa4351a2120b466d7` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:52.429569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e81b3f-f1da-4ac9-a0a2-9ea4392cea67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d38b517910894f7fa4351a2120b466d7)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:52.432108 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e81b3f-f1da-4ac9-a0a2-9ea4392cea67 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:52.550375 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-54e81b3f-f1da-4ac9-a0a2-9ea4392cea67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd38b517910894f7fa4351a2120b466d7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:52.562331 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-54e81b3f-f1da-4ac9-a0a2-9ea4392cea67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd38b517910894f7fa4351a2120b466d7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:52.563435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-54e81b3f-f1da-4ac9-a0a2-9ea4392cea67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd38b517910894f7fa4351a2120b466d7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:52.564126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-54e81b3f-f1da-4ac9-a0a2-9ea4392cea67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd38b517910894f7fa4351a2120b466d7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:52.577384 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:52.591737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-54e81b3f-f1da-4ac9-a0a2-9ea4392cea67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd38b517910894f7fa4351a2120b466d7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:52.657942 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1248/2498] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:35:52 2022] DELETE /identity/v3/users/d38b517910894f7fa4351a2120b466d7 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:53.570370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359f0f9-19cb-4ae0-989f-706c8055c020 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:53.599546 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359f0f9-19cb-4ae0-989f-706c8055c020 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:53.728962 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db53d953-453c-4fd8-97d4-94ad5aa1da9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:53.730032 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53d953-453c-4fd8-97d4-94ad5aa1da9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:53.730695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53d953-453c-4fd8-97d4-94ad5aa1da9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:53.734632 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db53d953-453c-4fd8-97d4-94ad5aa1da9f admin admin] PATH_INFO: `/v3/projects/d0c7b146e08f4a9ea11f1a73099eb7ea` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:53.739105 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db53d953-453c-4fd8-97d4-94ad5aa1da9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0c7b146e08f4a9ea11f1a73099eb7ea)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:53.741493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db53d953-453c-4fd8-97d4-94ad5aa1da9f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:53.854524 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-db53d953-453c-4fd8-97d4-94ad5aa1da9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0c7b146e08f4a9ea11f1a73099eb7ea', 'request_id': 'req-db53d953-453c-4fd8-97d4-94ad5aa1da9f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:53.875427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-db53d953-453c-4fd8-97d4-94ad5aa1da9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0c7b146e08f4a9ea11f1a73099eb7ea', 'request_id': 'req-db53d953-453c-4fd8-97d4-94ad5aa1da9f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:53.893807 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-db53d953-453c-4fd8-97d4-94ad5aa1da9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0c7b146e08f4a9ea11f1a73099eb7ea', 'request_id': 'req-db53d953-453c-4fd8-97d4-94ad5aa1da9f'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:53.919102 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e81b3f-f1da-4ac9-a0a2-9ea4392cea67 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:53.921272 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:53.922537 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-54e81b3f-f1da-4ac9-a0a2-9ea4392cea67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:53.934417 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-db53d953-453c-4fd8-97d4-94ad5aa1da9f admin admin] The token cache is being invalidate because project d0c7b146e08f4a9ea11f1a73099eb7ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:35:53.936214 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-db53d953-453c-4fd8-97d4-94ad5aa1da9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:53.939349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-db53d953-453c-4fd8-97d4-94ad5aa1da9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:53.948237 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1251/2499] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:35:53 2022] DELETE /identity/v3/projects/d0c7b146e08f4a9ea11f1a73099eb7ea => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:53.949810 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f69c5d-5787-45e9-9188-5ed2c8b31c15 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:53.950274 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f69c5d-5787-45e9-9188-5ed2c8b31c15 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:53.950888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f69c5d-5787-45e9-9188-5ed2c8b31c15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:54.000722 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-16f69c5d-5787-45e9-9188-5ed2c8b31c15 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:54.004403 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-16f69c5d-5787-45e9-9188-5ed2c8b31c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:54.010448 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e81b3f-f1da-4ac9-a0a2-9ea4392cea67 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:54.123027 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45d1be-1ab1-4cd7-b68d-4160816cfc72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:54.125239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45d1be-1ab1-4cd7-b68d-4160816cfc72 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:54.125594 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45d1be-1ab1-4cd7-b68d-4160816cfc72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:54.125836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d45d1be-1ab1-4cd7-b68d-4160816cfc72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:54.131562 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8d45d1be-1ab1-4cd7-b68d-4160816cfc72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:54.141071 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1252/2500] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:54.149169 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f69c5d-5787-45e9-9188-5ed2c8b31c15 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:54.162432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-16f69c5d-5787-45e9-9188-5ed2c8b31c15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:54.181048 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f69c5d-5787-45e9-9188-5ed2c8b31c15 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:54.186281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28ecff-9031-4f59-af7f-ddb28c355b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:54.188039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28ecff-9031-4f59-af7f-ddb28c355b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:54.188554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28ecff-9031-4f59-af7f-ddb28c355b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:54.188874 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28ecff-9031-4f59-af7f-ddb28c355b6a admin admin] PATH_INFO: `/v3/users/b7e13e8c76cb4757a42568112d16bd27` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:54.190718 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28ecff-9031-4f59-af7f-ddb28c355b6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7e13e8c76cb4757a42568112d16bd27)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:54.193350 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28ecff-9031-4f59-af7f-ddb28c355b6a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:54.303870 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4c28ecff-9031-4f59-af7f-ddb28c355b6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7e13e8c76cb4757a42568112d16bd27'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:54.321610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4c28ecff-9031-4f59-af7f-ddb28c355b6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7e13e8c76cb4757a42568112d16bd27'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:54.324816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45d1be-1ab1-4cd7-b68d-4160816cfc72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:54.326339 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d45d1be-1ab1-4cd7-b68d-4160816cfc72 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:54.332818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4c28ecff-9031-4f59-af7f-ddb28c355b6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7e13e8c76cb4757a42568112d16bd27'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:54.333733 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4c28ecff-9031-4f59-af7f-ddb28c355b6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7e13e8c76cb4757a42568112d16bd27'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:54.350066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4c28ecff-9031-4f59-af7f-ddb28c355b6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7e13e8c76cb4757a42568112d16bd27'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:54.357334 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1253/2501] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:35:54 2022] DELETE /identity/v3/users/b7e13e8c76cb4757a42568112d16bd27 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:54.359105 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e04ce-eb98-424d-8be0-851c325936ee None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:54.359462 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e04ce-eb98-424d-8be0-851c325936ee None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:54.359830 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e04ce-eb98-424d-8be0-851c325936ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:54.365794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1249/2502] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:53 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:35:54.384692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-593e04ce-eb98-424d-8be0-851c325936ee None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:54.386389 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-593e04ce-eb98-424d-8be0-851c325936ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:54.501058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1254/2503] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:54.506624 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45d1be-1ab1-4cd7-b68d-4160816cfc72 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:54.509680 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8d45d1be-1ab1-4cd7-b68d-4160816cfc72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:54.552444 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d45d1be-1ab1-4cd7-b68d-4160816cfc72 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:54.558110 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b10c0-69eb-4028-8fa1-22ee01bda52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:54.558744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b10c0-69eb-4028-8fa1-22ee01bda52b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:54.559030 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b10c0-69eb-4028-8fa1-22ee01bda52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:54.559312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3b10c0-69eb-4028-8fa1-22ee01bda52b admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:54.560028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b10c0-69eb-4028-8fa1-22ee01bda52b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:35:54.560837 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b10c0-69eb-4028-8fa1-22ee01bda52b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:54.562518 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3b10c0-69eb-4028-8fa1-22ee01bda52b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:54.623550 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1250/2504] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:35:54.626342 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e04ce-eb98-424d-8be0-851c325936ee None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:54.641277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e04ce-eb98-424d-8be0-851c325936ee None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:54.656608 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063faa6a-bad6-43fd-8470-7fa12a1d0cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:54.657300 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063faa6a-bad6-43fd-8470-7fa12a1d0cf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:54.657561 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063faa6a-bad6-43fd-8470-7fa12a1d0cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:54.657832 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063faa6a-bad6-43fd-8470-7fa12a1d0cf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:54.658519 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063faa6a-bad6-43fd-8470-7fa12a1d0cf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:54.661293 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063faa6a-bad6-43fd-8470-7fa12a1d0cf1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:54.696372 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:54.727860 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1255/2505] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:35:54 2022] POST /identity/v3/projects => generated 403 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:54.733183 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b10c0-69eb-4028-8fa1-22ee01bda52b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:54.778095 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3b10c0-69eb-4028-8fa1-22ee01bda52b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:54.857832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:54.868892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063faa6a-bad6-43fd-8470-7fa12a1d0cf1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:54.871370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-063faa6a-bad6-43fd-8470-7fa12a1d0cf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:54.901288 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b37949-a2c1-4ac5-bbea-f7354262acff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:54.902701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b37949-a2c1-4ac5-bbea-f7354262acff admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:54.903255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b37949-a2c1-4ac5-bbea-f7354262acff admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:54.903787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b37949-a2c1-4ac5-bbea-f7354262acff admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:54.905044 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b37949-a2c1-4ac5-bbea-f7354262acff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:54.908195 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b37949-a2c1-4ac5-bbea-f7354262acff admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:54.959263 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1251/2506] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:35:54 2022] POST /identity/v3/users => generated 302 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:54.963411 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b37949-a2c1-4ac5-bbea-f7354262acff admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:54.985229 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b37949-a2c1-4ac5-bbea-f7354262acff admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:55.010820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c079f-bd76-4ee1-8403-85759184afcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:55.012030 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c079f-bd76-4ee1-8403-85759184afcb admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:55.013225 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c079f-bd76-4ee1-8403-85759184afcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:55.013645 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8c079f-bd76-4ee1-8403-85759184afcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:55.014722 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c079f-bd76-4ee1-8403-85759184afcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:55.021756 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8c079f-bd76-4ee1-8403-85759184afcb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:55.046219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1252/2507] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:35:55.052166 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c079f-bd76-4ee1-8403-85759184afcb admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:55.064318 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063faa6a-bad6-43fd-8470-7fa12a1d0cf1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:55.068169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8c079f-bd76-4ee1-8403-85759184afcb admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:55.074262 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8715a1-18dd-4008-842e-020417d635a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:55.075424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8715a1-18dd-4008-842e-020417d635a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:55.075424 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8715a1-18dd-4008-842e-020417d635a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:55.075855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8715a1-18dd-4008-842e-020417d635a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:55.076886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8715a1-18dd-4008-842e-020417d635a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:55.079085 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8715a1-18dd-4008-842e-020417d635a7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:55.091890 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af5e86a-9df4-44c4-a34a-12a6f2684a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:55.093936 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5e86a-9df4-44c4-a34a-12a6f2684a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:55.094445 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5e86a-9df4-44c4-a34a-12a6f2684a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:55.094881 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af5e86a-9df4-44c4-a34a-12a6f2684a8c admin admin] PATH_INFO: `/v3/projects/9c4e68e56e744fa9af4631ae2e967a6d` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:55.099439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af5e86a-9df4-44c4-a34a-12a6f2684a8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c4e68e56e744fa9af4631ae2e967a6d)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:55.102531 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af5e86a-9df4-44c4-a34a-12a6f2684a8c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:55.111689 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1253/2508] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:55.113772 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8715a1-18dd-4008-842e-020417d635a7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:55.141119 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8715a1-18dd-4008-842e-020417d635a7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:55.153241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580952c-e710-47e1-b58e-f98395c14c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:55.153995 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580952c-e710-47e1-b58e-f98395c14c52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:55.154837 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580952c-e710-47e1-b58e-f98395c14c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:55.155121 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580952c-e710-47e1-b58e-f98395c14c52 admin admin] PATH_INFO: `/v3/projects/e6b45138bced4ec2aa62dac925390a39/users/541bdf7bbaa14750850bacbd7c5bed37/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:55.213366 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580952c-e710-47e1-b58e-f98395c14c52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6b45138bced4ec2aa62dac925390a39, user_id=541bdf7bbaa14750850bacbd7c5bed37, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:55.225446 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580952c-e710-47e1-b58e-f98395c14c52 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:55.250191 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7af5e86a-9df4-44c4-a34a-12a6f2684a8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c4e68e56e744fa9af4631ae2e967a6d', 'request_id': 'req-7af5e86a-9df4-44c4-a34a-12a6f2684a8c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:55.273380 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1254/2509] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:35:55 2022] PUT /identity/v3/projects/e6b45138bced4ec2aa62dac925390a39/users/541bdf7bbaa14750850bacbd7c5bed37/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:55.278711 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72a292-5933-4405-82bc-35e84a6c0f8d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:55.279223 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72a292-5933-4405-82bc-35e84a6c0f8d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:55.279618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea72a292-5933-4405-82bc-35e84a6c0f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:55.281431 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7af5e86a-9df4-44c4-a34a-12a6f2684a8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c4e68e56e744fa9af4631ae2e967a6d', 'request_id': 'req-7af5e86a-9df4-44c4-a34a-12a6f2684a8c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:55.294070 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7af5e86a-9df4-44c4-a34a-12a6f2684a8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c4e68e56e744fa9af4631ae2e967a6d', 'request_id': 'req-7af5e86a-9df4-44c4-a34a-12a6f2684a8c'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:55.307428 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ea72a292-5933-4405-82bc-35e84a6c0f8d None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:55.309086 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ea72a292-5933-4405-82bc-35e84a6c0f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:55.317134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7af5e86a-9df4-44c4-a34a-12a6f2684a8c admin admin] The token cache is being invalidate because project 9c4e68e56e744fa9af4631ae2e967a6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:35:55.317854 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7af5e86a-9df4-44c4-a34a-12a6f2684a8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:55.318991 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-7af5e86a-9df4-44c4-a34a-12a6f2684a8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:55.324959 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1256/2510] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:35:54 2022] DELETE /identity/v3/projects/9c4e68e56e744fa9af4631ae2e967a6d => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:55.326617 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b992d-c6ea-4d48-834c-9eb64933ab4a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:55.326914 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b992d-c6ea-4d48-834c-9eb64933ab4a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:55.327197 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b992d-c6ea-4d48-834c-9eb64933ab4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:55.351910 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-fe0b992d-c6ea-4d48-834c-9eb64933ab4a None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:55.353554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fe0b992d-c6ea-4d48-834c-9eb64933ab4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:55.446553 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1255/2511] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:55.451371 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1257/2512] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:55.455191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1603b7-0468-4820-90d2-d532fc3c8f49 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:55.455482 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1603b7-0468-4820-90d2-d532fc3c8f49 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:55.455788 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1603b7-0468-4820-90d2-d532fc3c8f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:55.462388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b992d-c6ea-4d48-834c-9eb64933ab4a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:55.464618 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fe0b992d-c6ea-4d48-834c-9eb64933ab4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:55.494754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4b1603b7-0468-4820-90d2-d532fc3c8f49 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:55.495992 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4b1603b7-0468-4820-90d2-d532fc3c8f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:55.520500 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1256/2513] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:35:55.525362 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1603b7-0468-4820-90d2-d532fc3c8f49 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:55.528506 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:55.544303 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1603b7-0468-4820-90d2-d532fc3c8f49 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:55.558579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbafbc4-126d-4931-b5c1-64a73ada2aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:55.559225 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbafbc4-126d-4931-b5c1-64a73ada2aba admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:55.559572 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbafbc4-126d-4931-b5c1-64a73ada2aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:55.559822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbafbc4-126d-4931-b5c1-64a73ada2aba admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:55.560532 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbafbc4-126d-4931-b5c1-64a73ada2aba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:35:55.560838 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbafbc4-126d-4931-b5c1-64a73ada2aba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:55.562493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbafbc4-126d-4931-b5c1-64a73ada2aba admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:55.590070 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1257/2514] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:35:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:35:55.594126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbafbc4-126d-4931-b5c1-64a73ada2aba admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:55.620172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbafbc4-126d-4931-b5c1-64a73ada2aba admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:55.630278 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed94f5c-834f-4546-81b6-04d2bbc556f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:55.630890 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed94f5c-834f-4546-81b6-04d2bbc556f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:55.631169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed94f5c-834f-4546-81b6-04d2bbc556f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:55.631438 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed94f5c-834f-4546-81b6-04d2bbc556f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:55.632111 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed94f5c-834f-4546-81b6-04d2bbc556f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:55.640526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed94f5c-834f-4546-81b6-04d2bbc556f1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:55.651318 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b992d-c6ea-4d48-834c-9eb64933ab4a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:55.663406 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337a4d4-5f18-46f2-a470-c039ed22767b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:55.664213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337a4d4-5f18-46f2-a470-c039ed22767b service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:55.664596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337a4d4-5f18-46f2-a470-c039ed22767b service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:55.698289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337a4d4-5f18-46f2-a470-c039ed22767b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:55.736110 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337a4d4-5f18-46f2-a470-c039ed22767b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:55.739224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337a4d4-5f18-46f2-a470-c039ed22767b service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:55.756673 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1258/2515] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:35:55 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:55.758792 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed94f5c-834f-4546-81b6-04d2bbc556f1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:55.790895 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1258/2516] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:35:55.853657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:55.914444 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed94f5c-834f-4546-81b6-04d2bbc556f1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:55.952111 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc85325-82fb-463d-b3b7-373e9700cd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:55.957836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc85325-82fb-463d-b3b7-373e9700cd42 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:55.958317 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc85325-82fb-463d-b3b7-373e9700cd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:55.960769 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc85325-82fb-463d-b3b7-373e9700cd42 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:55.962177 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc85325-82fb-463d-b3b7-373e9700cd42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:55.965267 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc85325-82fb-463d-b3b7-373e9700cd42 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:56.009420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337a4d4-5f18-46f2-a470-c039ed22767b service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:56.021730 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1259/2517] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:35:55 2022] POST /identity/v3/users => generated 287 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:35:56.024505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337a4d4-5f18-46f2-a470-c039ed22767b service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:56.031501 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcc8bf-3a80-45cc-b824-ed810e26d9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:56.032446 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcc8bf-3a80-45cc-b824-ed810e26d9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:56.032847 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcc8bf-3a80-45cc-b824-ed810e26d9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:56.033196 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcc8bf-3a80-45cc-b824-ed810e26d9d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:56.034661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcc8bf-3a80-45cc-b824-ed810e26d9d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:56.036929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcc8bf-3a80-45cc-b824-ed810e26d9d0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:56.058274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc85325-82fb-463d-b3b7-373e9700cd42 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:56.080594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1259/2518] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:56.090747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc85325-82fb-463d-b3b7-373e9700cd42 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:56.110832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46172486-1067-4fb1-ba83-ad1a4660a6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:56.111630 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46172486-1067-4fb1-ba83-ad1a4660a6f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:56.113266 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46172486-1067-4fb1-ba83-ad1a4660a6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:56.114111 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46172486-1067-4fb1-ba83-ad1a4660a6f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:56.115032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46172486-1067-4fb1-ba83-ad1a4660a6f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:56.117498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46172486-1067-4fb1-ba83-ad1a4660a6f6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:56.134233 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1260/2519] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:56 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:56.138564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcc8bf-3a80-45cc-b824-ed810e26d9d0 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:56.155185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcc8bf-3a80-45cc-b824-ed810e26d9d0 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:56.161931 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375c7b8-2ee8-49e5-96b2-0e40fcd3b86e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:56.162599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375c7b8-2ee8-49e5-96b2-0e40fcd3b86e admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:56.162882 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375c7b8-2ee8-49e5-96b2-0e40fcd3b86e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:56.163155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375c7b8-2ee8-49e5-96b2-0e40fcd3b86e admin admin] PATH_INFO: `/v3/projects/3dc1c6397c6a499e989953d4cd8f6764/users/133c7e259d3140a487a4c7d14c736a67/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:56.182952 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9375c7b8-2ee8-49e5-96b2-0e40fcd3b86e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3dc1c6397c6a499e989953d4cd8f6764, user_id=133c7e259d3140a487a4c7d14c736a67, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:56.186149 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9375c7b8-2ee8-49e5-96b2-0e40fcd3b86e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:56.203327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1260/2520] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:35:56 2022] PUT /identity/v3/projects/3dc1c6397c6a499e989953d4cd8f6764/users/133c7e259d3140a487a4c7d14c736a67/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:56.215425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb61b5-54f3-4817-9fd3-dc01fd990c80 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:56.215735 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb61b5-54f3-4817-9fd3-dc01fd990c80 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:56.216029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fb61b5-54f3-4817-9fd3-dc01fd990c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:56.270880 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-55fb61b5-54f3-4817-9fd3-dc01fd990c80 None None] MFA Rules not processed for user `133c7e259d3140a487a4c7d14c736a67`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:56.279833 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-55fb61b5-54f3-4817-9fd3-dc01fd990c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:56.382825 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1261/2521] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:56.390241 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375c7b8-2ee8-49e5-96b2-0e40fcd3b86e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:56.392975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9375c7b8-2ee8-49e5-96b2-0e40fcd3b86e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:56.560911 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9375c7b8-2ee8-49e5-96b2-0e40fcd3b86e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:56.568348 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137dae0f-956b-47a3-8827-ca974bd8d79e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:56.569109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137dae0f-956b-47a3-8827-ca974bd8d79e service nova] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:56.569518 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137dae0f-956b-47a3-8827-ca974bd8d79e service nova] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:56.570033 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137dae0f-956b-47a3-8827-ca974bd8d79e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:56.580040 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137dae0f-956b-47a3-8827-ca974bd8d79e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:56.582236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137dae0f-956b-47a3-8827-ca974bd8d79e service nova] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:56.601202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1261/2522] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:35:56.693376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ae289-d3f9-4e77-b702-8dc8f84198d9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:56.693752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ae289-d3f9-4e77-b702-8dc8f84198d9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:56.694043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736ae289-d3f9-4e77-b702-8dc8f84198d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:56.722225 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-736ae289-d3f9-4e77-b702-8dc8f84198d9 None None] MFA Rules not processed for user `85626350279c40ea99da93b9ed6994b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:56.723463 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-736ae289-d3f9-4e77-b702-8dc8f84198d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:56.793720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1262/2523] 10.222.0.39 () {58 vars in 967 bytes} [Thu Jul 28 22:35:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:56.806552 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137dae0f-956b-47a3-8827-ca974bd8d79e service nova] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:56.808685 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-137dae0f-956b-47a3-8827-ca974bd8d79e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:56.896448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137dae0f-956b-47a3-8827-ca974bd8d79e service nova] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:56.904581 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ab10b-501a-45e3-b44e-796d5a2792f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9107872f147f6ab8698b67bb545af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:56.905660 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ab10b-501a-45e3-b44e-796d5a2792f3 service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:56.906125 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ab10b-501a-45e3-b44e-796d5a2792f3 service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:56.906600 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ab10b-501a-45e3-b44e-796d5a2792f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:56.923671 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ef317-0eaa-4442-aecf-d7817c6199ee None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:56.923987 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ef317-0eaa-4442-aecf-d7817c6199ee None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:56.924451 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2ef317-0eaa-4442-aecf-d7817c6199ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:56.932517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ab10b-501a-45e3-b44e-796d5a2792f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:56.934616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ab10b-501a-45e3-b44e-796d5a2792f3 service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:56.962881 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1262/2524] 10.222.0.39 () {60 vars in 1384 bytes} [Thu Jul 28 22:35:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 22:35:56.976670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0d2ef317-0eaa-4442-aecf-d7817c6199ee None None] MFA Rules not processed for user `541bdf7bbaa14750850bacbd7c5bed37`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:56.977916 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0d2ef317-0eaa-4442-aecf-d7817c6199ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:57.063048 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:57.106052 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1263/2525] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:56 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:35:57.110200 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ab10b-501a-45e3-b44e-796d5a2792f3 service swift] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:57.114400 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-818ab10b-501a-45e3-b44e-796d5a2792f3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:57.160280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:57.320595 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ab10b-501a-45e3-b44e-796d5a2792f3 service swift] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:57.350054 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6623df-9ff2-4c81-b133-abede59853b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:57.360573 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6623df-9ff2-4c81-b133-abede59853b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:57.361592 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6623df-9ff2-4c81-b133-abede59853b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:57.362349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6623df-9ff2-4c81-b133-abede59853b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:57.363284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6623df-9ff2-4c81-b133-abede59853b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:57.367195 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6623df-9ff2-4c81-b133-abede59853b8 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:57.411998 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1263/2526] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:35:57 2022] POST /identity/v3/projects => generated 405 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:35:57.416486 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ef317-0eaa-4442-aecf-d7817c6199ee None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:57.435459 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2ef317-0eaa-4442-aecf-d7817c6199ee None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:57.541053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6623df-9ff2-4c81-b133-abede59853b8 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:57.575480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985c806-2b8a-40a6-907c-4031def1287f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:57.576223 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985c806-2b8a-40a6-907c-4031def1287f admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:57.576903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985c806-2b8a-40a6-907c-4031def1287f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:57.577250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9985c806-2b8a-40a6-907c-4031def1287f admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:57.578040 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985c806-2b8a-40a6-907c-4031def1287f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:57.580149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9985c806-2b8a-40a6-907c-4031def1287f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:57.613496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1264/2527] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:35:57 2022] POST /identity/v3/users => generated 309 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:35:57.617844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985c806-2b8a-40a6-907c-4031def1287f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:57.638189 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9985c806-2b8a-40a6-907c-4031def1287f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:57.644409 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa80f68-16f5-464d-bc92-44eb6a08fda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:57.645287 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa80f68-16f5-464d-bc92-44eb6a08fda6 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:57.645767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa80f68-16f5-464d-bc92-44eb6a08fda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:57.646264 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa80f68-16f5-464d-bc92-44eb6a08fda6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:57.647158 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa80f68-16f5-464d-bc92-44eb6a08fda6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:57.649395 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa80f68-16f5-464d-bc92-44eb6a08fda6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:57.655322 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6623df-9ff2-4c81-b133-abede59853b8 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:57.666702 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1265/2528] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:57.666841 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb000a-4025-431a-912a-160ea62e30fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:57.668334 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb000a-4025-431a-912a-160ea62e30fe service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:57.668627 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb000a-4025-431a-912a-160ea62e30fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:57.668991 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb000a-4025-431a-912a-160ea62e30fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:57.671272 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa80f68-16f5-464d-bc92-44eb6a08fda6 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:57.673726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6ceb000a-4025-431a-912a-160ea62e30fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:57.682668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa80f68-16f5-464d-bc92-44eb6a08fda6 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:57.688816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f1826-d006-44f1-89a0-934c0214bec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:57.689511 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f1826-d006-44f1-89a0-934c0214bec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:57.689671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb000a-4025-431a-912a-160ea62e30fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:57.689863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f1826-d006-44f1-89a0-934c0214bec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:57.691393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f1826-d006-44f1-89a0-934c0214bec0 admin admin] PATH_INFO: `/v3/projects/344f10a169954071b3023faad8b1c456/users/7b1037d0d805482c871c2f0ca4b00813/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:57.691800 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb000a-4025-431a-912a-160ea62e30fe service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:57.707200 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1264/2529] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:35:57.713705 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f1826-d006-44f1-89a0-934c0214bec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=344f10a169954071b3023faad8b1c456, user_id=7b1037d0d805482c871c2f0ca4b00813, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:57.721127 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f1826-d006-44f1-89a0-934c0214bec0 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:57.758782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1266/2530] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:35:57 2022] PUT /identity/v3/projects/344f10a169954071b3023faad8b1c456/users/7b1037d0d805482c871c2f0ca4b00813/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:35:57.762827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb000a-4025-431a-912a-160ea62e30fe service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:57.773417 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb000a-4025-431a-912a-160ea62e30fe service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:57.855254 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1e66b7-84bb-4d4c-8d75-bb1f4488f290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:57.856171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e66b7-84bb-4d4c-8d75-bb1f4488f290 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:57.856659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e66b7-84bb-4d4c-8d75-bb1f4488f290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:57.857018 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1e66b7-84bb-4d4c-8d75-bb1f4488f290 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:57.858357 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e66b7-84bb-4d4c-8d75-bb1f4488f290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:57.860214 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1e66b7-84bb-4d4c-8d75-bb1f4488f290 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:57.878589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1265/2531] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:35:57.882706 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f1826-d006-44f1-89a0-934c0214bec0 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:57.900030 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f1826-d006-44f1-89a0-934c0214bec0 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:57.905517 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a098b-2615-48b6-b76f-376dcef0f775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:57.906310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a098b-2615-48b6-b76f-376dcef0f775 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:57.906692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a098b-2615-48b6-b76f-376dcef0f775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:57.906996 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2a098b-2615-48b6-b76f-376dcef0f775 admin admin] PATH_INFO: `/v3/projects/344f10a169954071b3023faad8b1c456/users/7b1037d0d805482c871c2f0ca4b00813/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:57.910361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a098b-2615-48b6-b76f-376dcef0f775 admin admin] RBAC: Authorizing `identity:create_grant(project_id=344f10a169954071b3023faad8b1c456, user_id=7b1037d0d805482c871c2f0ca4b00813, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:57.913156 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2a098b-2615-48b6-b76f-376dcef0f775 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:57.925151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1267/2532] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:35:57 2022] PUT /identity/v3/projects/344f10a169954071b3023faad8b1c456/users/7b1037d0d805482c871c2f0ca4b00813/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:35:59.047457 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf4526-bb1e-4c04-b09e-ee9306deb224 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:59.048026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf4526-bb1e-4c04-b09e-ee9306deb224 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:59.048026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf4526-bb1e-4c04-b09e-ee9306deb224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:59.090650 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b8cf4526-bb1e-4c04-b09e-ee9306deb224 None None] MFA Rules not processed for user `7b1037d0d805482c871c2f0ca4b00813`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:59.092295 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b8cf4526-bb1e-4c04-b09e-ee9306deb224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:59.116649 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:59.199805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1266/2533] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:59 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:59.526250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a098b-2615-48b6-b76f-376dcef0f775 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:59.601004 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2a098b-2615-48b6-b76f-376dcef0f775 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:59.605863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203ab54-dffe-49fb-8b21-b1db81eecac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:59.606756 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203ab54-dffe-49fb-8b21-b1db81eecac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:59.606952 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203ab54-dffe-49fb-8b21-b1db81eecac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:59.607821 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203ab54-dffe-49fb-8b21-b1db81eecac6 admin admin] PATH_INFO: `/v3/users/133c7e259d3140a487a4c7d14c736a67` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:59.608987 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2203ab54-dffe-49fb-8b21-b1db81eecac6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=133c7e259d3140a487a4c7d14c736a67)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:59.610934 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2203ab54-dffe-49fb-8b21-b1db81eecac6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:59.637970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:35:59.669457 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2203ab54-dffe-49fb-8b21-b1db81eecac6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '133c7e259d3140a487a4c7d14c736a67'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:59.682529 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2203ab54-dffe-49fb-8b21-b1db81eecac6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '133c7e259d3140a487a4c7d14c736a67'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:59.686130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2203ab54-dffe-49fb-8b21-b1db81eecac6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '133c7e259d3140a487a4c7d14c736a67'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:59.687599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2203ab54-dffe-49fb-8b21-b1db81eecac6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '133c7e259d3140a487a4c7d14c736a67'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:59.706409 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2203ab54-dffe-49fb-8b21-b1db81eecac6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '133c7e259d3140a487a4c7d14c736a67'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:35:59.727204 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1268/2534] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:35:59 2022] DELETE /identity/v3/users/133c7e259d3140a487a4c7d14c736a67 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:35:59.732532 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e597037-a631-45b9-96db-16cbe082b865 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:59.732690 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e597037-a631-45b9-96db-16cbe082b865 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:59.733172 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e597037-a631-45b9-96db-16cbe082b865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:59.756521 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5e597037-a631-45b9-96db-16cbe082b865 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:35:59.757892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5e597037-a631-45b9-96db-16cbe082b865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:59.838345 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1267/2535] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:35:59.858165 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203ab54-dffe-49fb-8b21-b1db81eecac6 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:35:59.860771 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2203ab54-dffe-49fb-8b21-b1db81eecac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:35:59.942771 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203ab54-dffe-49fb-8b21-b1db81eecac6 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:35:59.947922 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b64e5-cd24-4d52-a8f2-27be476db5bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:35:59.948578 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b64e5-cd24-4d52-a8f2-27be476db5bd service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:35:59.948862 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b64e5-cd24-4d52-a8f2-27be476db5bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:35:59.949136 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480b64e5-cd24-4d52-a8f2-27be476db5bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:35:59.961852 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b64e5-cd24-4d52-a8f2-27be476db5bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:35:59.963346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480b64e5-cd24-4d52-a8f2-27be476db5bd service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:35:59.991758 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1269/2536] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:35:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:00.282673 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e597037-a631-45b9-96db-16cbe082b865 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:00.286443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5e597037-a631-45b9-96db-16cbe082b865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:00.349933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e597037-a631-45b9-96db-16cbe082b865 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:00.360704 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b4eba-409b-4d86-bb64-b94fa91c032e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:00.362596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b4eba-409b-4d86-bb64-b94fa91c032e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:00.363169 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b4eba-409b-4d86-bb64-b94fa91c032e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:00.363657 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3b4eba-409b-4d86-bb64-b94fa91c032e admin admin] PATH_INFO: `/v3/users/d649218b6ae047c480de9640727aaded` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:00.366663 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3b4eba-409b-4d86-bb64-b94fa91c032e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d649218b6ae047c480de9640727aaded)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:00.371635 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3b4eba-409b-4d86-bb64-b94fa91c032e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:00.454373 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fd3b4eba-409b-4d86-bb64-b94fa91c032e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd649218b6ae047c480de9640727aaded'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:00.467994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fd3b4eba-409b-4d86-bb64-b94fa91c032e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd649218b6ae047c480de9640727aaded'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:00.478257 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fd3b4eba-409b-4d86-bb64-b94fa91c032e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd649218b6ae047c480de9640727aaded'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:00.479085 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fd3b4eba-409b-4d86-bb64-b94fa91c032e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd649218b6ae047c480de9640727aaded'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:00.515970 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fd3b4eba-409b-4d86-bb64-b94fa91c032e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd649218b6ae047c480de9640727aaded'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:00.530379 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1268/2537] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:00 2022] DELETE /identity/v3/users/d649218b6ae047c480de9640727aaded => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:00.742784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b64e5-cd24-4d52-a8f2-27be476db5bd service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:00.744282 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-480b64e5-cd24-4d52-a8f2-27be476db5bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:00.844042 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480b64e5-cd24-4d52-a8f2-27be476db5bd service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:00.863388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b4eba-409b-4d86-bb64-b94fa91c032e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:00.864951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fd3b4eba-409b-4d86-bb64-b94fa91c032e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:00.880994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:00.958624 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669989c9-0600-463a-8afb-52a5c430d3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:00.959865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669989c9-0600-463a-8afb-52a5c430d3c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:00.960527 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669989c9-0600-463a-8afb-52a5c430d3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:00.961123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669989c9-0600-463a-8afb-52a5c430d3c7 admin admin] PATH_INFO: `/v3/projects/3dc1c6397c6a499e989953d4cd8f6764` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:00.962841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669989c9-0600-463a-8afb-52a5c430d3c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3dc1c6397c6a499e989953d4cd8f6764)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:00.965185 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669989c9-0600-463a-8afb-52a5c430d3c7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:00.968804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:01.069263 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-669989c9-0600-463a-8afb-52a5c430d3c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3dc1c6397c6a499e989953d4cd8f6764', 'request_id': 'req-669989c9-0600-463a-8afb-52a5c430d3c7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:01.077445 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-669989c9-0600-463a-8afb-52a5c430d3c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3dc1c6397c6a499e989953d4cd8f6764', 'request_id': 'req-669989c9-0600-463a-8afb-52a5c430d3c7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:01.092765 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-669989c9-0600-463a-8afb-52a5c430d3c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3dc1c6397c6a499e989953d4cd8f6764', 'request_id': 'req-669989c9-0600-463a-8afb-52a5c430d3c7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:01.094451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3b4eba-409b-4d86-bb64-b94fa91c032e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:01.120104 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-669989c9-0600-463a-8afb-52a5c430d3c7 admin admin] The token cache is being invalidate because project 3dc1c6397c6a499e989953d4cd8f6764 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:01.121395 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-669989c9-0600-463a-8afb-52a5c430d3c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:01.122120 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-669989c9-0600-463a-8afb-52a5c430d3c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:01.124228 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1270/2538] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:00 2022] DELETE /identity/v3/projects/3dc1c6397c6a499e989953d4cd8f6764 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:01.125114 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669989c9-0600-463a-8afb-52a5c430d3c7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:01.126542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-669989c9-0600-463a-8afb-52a5c430d3c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:01.185669 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0453f867-7888-45a7-b627-9ab1ffe0f90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:01.186318 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453f867-7888-45a7-b627-9ab1ffe0f90e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:01.186999 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453f867-7888-45a7-b627-9ab1ffe0f90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:01.187427 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0453f867-7888-45a7-b627-9ab1ffe0f90e admin admin] PATH_INFO: `/v3/users/541bdf7bbaa14750850bacbd7c5bed37` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:01.190432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453f867-7888-45a7-b627-9ab1ffe0f90e admin admin] RBAC: Authorizing `identity:delete_user(user_id=541bdf7bbaa14750850bacbd7c5bed37)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:01.192085 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0453f867-7888-45a7-b627-9ab1ffe0f90e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:01.231543 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669989c9-0600-463a-8afb-52a5c430d3c7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:01.236907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1bdd1e-50f9-4f6c-83ae-d39795a23375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:01.237996 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bdd1e-50f9-4f6c-83ae-d39795a23375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:01.238243 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bdd1e-50f9-4f6c-83ae-d39795a23375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:01.238552 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1bdd1e-50f9-4f6c-83ae-d39795a23375 admin admin] PATH_INFO: `/v3/projects/f5144f6ac53848fe84bdcb15e5242f1d` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:01.239888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1bdd1e-50f9-4f6c-83ae-d39795a23375 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5144f6ac53848fe84bdcb15e5242f1d)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:01.241968 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1bdd1e-50f9-4f6c-83ae-d39795a23375 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:01.259363 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0453f867-7888-45a7-b627-9ab1ffe0f90e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '541bdf7bbaa14750850bacbd7c5bed37'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:01.268139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0453f867-7888-45a7-b627-9ab1ffe0f90e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '541bdf7bbaa14750850bacbd7c5bed37'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:01.281520 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0453f867-7888-45a7-b627-9ab1ffe0f90e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '541bdf7bbaa14750850bacbd7c5bed37'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:01.282556 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0453f867-7888-45a7-b627-9ab1ffe0f90e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '541bdf7bbaa14750850bacbd7c5bed37'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:01.298570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-0453f867-7888-45a7-b627-9ab1ffe0f90e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '541bdf7bbaa14750850bacbd7c5bed37'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:01.301793 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:01.308387 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1269/2539] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:00 2022] DELETE /identity/v3/users/541bdf7bbaa14750850bacbd7c5bed37 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:01.313238 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c09630-ca5d-4a9d-aef7-ad89b0ba5480 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:01.313673 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c09630-ca5d-4a9d-aef7-ad89b0ba5480 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:01.314076 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c09630-ca5d-4a9d-aef7-ad89b0ba5480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:01.325266 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9b1bdd1e-50f9-4f6c-83ae-d39795a23375 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5144f6ac53848fe84bdcb15e5242f1d', 'request_id': 'req-9b1bdd1e-50f9-4f6c-83ae-d39795a23375'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:01.338273 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9b1bdd1e-50f9-4f6c-83ae-d39795a23375 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5144f6ac53848fe84bdcb15e5242f1d', 'request_id': 'req-9b1bdd1e-50f9-4f6c-83ae-d39795a23375'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:01.354763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9b1bdd1e-50f9-4f6c-83ae-d39795a23375 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5144f6ac53848fe84bdcb15e5242f1d', 'request_id': 'req-9b1bdd1e-50f9-4f6c-83ae-d39795a23375'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:01.383656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9b1bdd1e-50f9-4f6c-83ae-d39795a23375 admin admin] The token cache is being invalidate because project f5144f6ac53848fe84bdcb15e5242f1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:01.383955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9b1bdd1e-50f9-4f6c-83ae-d39795a23375 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:01.385834 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9b1bdd1e-50f9-4f6c-83ae-d39795a23375 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:01.391335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1271/2540] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:01 2022] DELETE /identity/v3/projects/f5144f6ac53848fe84bdcb15e5242f1d => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:01.396793 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ab83c-7ccd-4dc2-9117-74e9f722fa29 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:01.397109 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ab83c-7ccd-4dc2-9117-74e9f722fa29 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:01.397637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ab83c-7ccd-4dc2-9117-74e9f722fa29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:01.398950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f8c09630-ca5d-4a9d-aef7-ad89b0ba5480 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:01.400383 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f8c09630-ca5d-4a9d-aef7-ad89b0ba5480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:01.439772 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-bb3ab83c-7ccd-4dc2-9117-74e9f722fa29 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:01.442214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bb3ab83c-7ccd-4dc2-9117-74e9f722fa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:01.471039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:01.484010 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1270/2541] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:01.494300 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b3ac6-1964-4d9d-b276-bf3665348dd2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:01.494824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b3ac6-1964-4d9d-b276-bf3665348dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:01.496139 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2b3ac6-1964-4d9d-b276-bf3665348dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:01.522239 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-8f2b3ac6-1964-4d9d-b276-bf3665348dd2 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:01.523694 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-8f2b3ac6-1964-4d9d-b276-bf3665348dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:01.533573 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1271/2542] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:36:01.535724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1272/2543] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:36:01.538660 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b3ac6-1964-4d9d-b276-bf3665348dd2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:01.543421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a28d9-bcc7-498a-b1fc-c87ce0c956d0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:01.543903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a28d9-bcc7-498a-b1fc-c87ce0c956d0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:01.544171 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4a28d9-bcc7-498a-b1fc-c87ce0c956d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:01.567975 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3b4a28d9-bcc7-498a-b1fc-c87ce0c956d0 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:01.569186 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3b4a28d9-bcc7-498a-b1fc-c87ce0c956d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:01.569570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2b3ac6-1964-4d9d-b276-bf3665348dd2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:01.579364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1273/2544] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:01.584449 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a28d9-bcc7-498a-b1fc-c87ce0c956d0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:01.585335 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1056c0-4282-4fb1-8ad4-6c2f4b877749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:01.586336 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1056c0-4282-4fb1-8ad4-6c2f4b877749 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:01.587306 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3b4a28d9-bcc7-498a-b1fc-c87ce0c956d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:01.587640 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1056c0-4282-4fb1-8ad4-6c2f4b877749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:01.589060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1056c0-4282-4fb1-8ad4-6c2f4b877749 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:01.589819 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1056c0-4282-4fb1-8ad4-6c2f4b877749 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:36:01.590168 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1056c0-4282-4fb1-8ad4-6c2f4b877749 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:01.591814 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1056c0-4282-4fb1-8ad4-6c2f4b877749 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:01.604565 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4a28d9-bcc7-498a-b1fc-c87ce0c956d0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:01.611815 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1272/2545] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:36:01.612970 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1056c0-4282-4fb1-8ad4-6c2f4b877749 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:01.617348 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a52438-275d-4b22-a86e-9d51a1a66260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:01.618071 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a52438-275d-4b22-a86e-9d51a1a66260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:01.618354 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a52438-275d-4b22-a86e-9d51a1a66260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:01.618944 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a52438-275d-4b22-a86e-9d51a1a66260 admin admin] PATH_INFO: `/v3/projects/e6b45138bced4ec2aa62dac925390a39` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:01.620049 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a52438-275d-4b22-a86e-9d51a1a66260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6b45138bced4ec2aa62dac925390a39)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:01.621901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a52438-275d-4b22-a86e-9d51a1a66260 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:01.624280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1056c0-4282-4fb1-8ad4-6c2f4b877749 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:01.631468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1c49d-02b0-4efa-abb1-5ac71d147dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:01.632114 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1c49d-02b0-4efa-abb1-5ac71d147dcc admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:01.632397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1c49d-02b0-4efa-abb1-5ac71d147dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:01.633099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd1c49d-02b0-4efa-abb1-5ac71d147dcc admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:01.634014 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1c49d-02b0-4efa-abb1-5ac71d147dcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:36:01.634187 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1c49d-02b0-4efa-abb1-5ac71d147dcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:01.635796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd1c49d-02b0-4efa-abb1-5ac71d147dcc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:01.651313 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1273/2546] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:36:01.652169 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1c49d-02b0-4efa-abb1-5ac71d147dcc admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:01.665449 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd1c49d-02b0-4efa-abb1-5ac71d147dcc admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:01.670102 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b5874-da5f-4cf7-b053-f6e9fec3354a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:01.670721 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b5874-da5f-4cf7-b053-f6e9fec3354a admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:01.671004 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b5874-da5f-4cf7-b053-f6e9fec3354a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:01.671279 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9b5874-da5f-4cf7-b053-f6e9fec3354a admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:01.672631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b5874-da5f-4cf7-b053-f6e9fec3354a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:01.675730 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9b5874-da5f-4cf7-b053-f6e9fec3354a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:01.685301 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c8a52438-275d-4b22-a86e-9d51a1a66260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6b45138bced4ec2aa62dac925390a39', 'request_id': 'req-c8a52438-275d-4b22-a86e-9d51a1a66260'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:01.695108 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c8a52438-275d-4b22-a86e-9d51a1a66260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6b45138bced4ec2aa62dac925390a39', 'request_id': 'req-c8a52438-275d-4b22-a86e-9d51a1a66260'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:01.705034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c8a52438-275d-4b22-a86e-9d51a1a66260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6b45138bced4ec2aa62dac925390a39', 'request_id': 'req-c8a52438-275d-4b22-a86e-9d51a1a66260'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:01.724632 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1274/2547] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:01 2022] POST /identity/v3/projects => generated 387 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:01.726462 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b5874-da5f-4cf7-b053-f6e9fec3354a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:01.732287 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c8a52438-275d-4b22-a86e-9d51a1a66260 admin admin] The token cache is being invalidate because project e6b45138bced4ec2aa62dac925390a39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:01.732589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c8a52438-275d-4b22-a86e-9d51a1a66260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:01.733292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c8a52438-275d-4b22-a86e-9d51a1a66260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:01.735144 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1274/2548] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:01 2022] DELETE /identity/v3/projects/e6b45138bced4ec2aa62dac925390a39 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:01.736123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a52438-275d-4b22-a86e-9d51a1a66260 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:01.737803 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c8a52438-275d-4b22-a86e-9d51a1a66260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:01.812712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a52438-275d-4b22-a86e-9d51a1a66260 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:01.830370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9b5874-da5f-4cf7-b053-f6e9fec3354a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:01.837177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffee1fe-f6ac-4633-9bba-8c23e6424b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:01.837972 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffee1fe-f6ac-4633-9bba-8c23e6424b3c admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:01.838411 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffee1fe-f6ac-4633-9bba-8c23e6424b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:01.838756 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffee1fe-f6ac-4633-9bba-8c23e6424b3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:01.841469 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffee1fe-f6ac-4633-9bba-8c23e6424b3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:01.842541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffee1fe-f6ac-4633-9bba-8c23e6424b3c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:01.865944 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:01.882636 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:01.886560 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4e2d3-ef5d-4838-a248-3121d856ce6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:01.887982 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4e2d3-ef5d-4838-a248-3121d856ce6a admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:01.888921 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4e2d3-ef5d-4838-a248-3121d856ce6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:01.888921 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b4e2d3-ef5d-4838-a248-3121d856ce6a admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:01.889911 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4e2d3-ef5d-4838-a248-3121d856ce6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:01.895676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b4e2d3-ef5d-4838-a248-3121d856ce6a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:01.903155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1275/2549] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:01 2022] POST /identity/v3/projects => generated 363 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:01.904222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffee1fe-f6ac-4633-9bba-8c23e6424b3c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:01.909448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2ffee1fe-f6ac-4633-9bba-8c23e6424b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:01.938239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1275/2550] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:01 2022] POST /identity/v3/users => generated 294 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:01.939173 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4e2d3-ef5d-4838-a248-3121d856ce6a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:01.943819 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f0b4e2d3-ef5d-4838-a248-3121d856ce6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:01.982758 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b4e2d3-ef5d-4838-a248-3121d856ce6a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:02.082662 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffee1fe-f6ac-4633-9bba-8c23e6424b3c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:02.090966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b10c1-0ad9-48ac-870b-6aeef280a052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:02.092199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b10c1-0ad9-48ac-870b-6aeef280a052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:02.093172 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b10c1-0ad9-48ac-870b-6aeef280a052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:02.093279 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209b10c1-0ad9-48ac-870b-6aeef280a052 admin admin] PATH_INFO: `/v3/users/7b1037d0d805482c871c2f0ca4b00813` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:02.096160 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b10c1-0ad9-48ac-870b-6aeef280a052 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b1037d0d805482c871c2f0ca4b00813)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:02.108957 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209b10c1-0ad9-48ac-870b-6aeef280a052 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:02.136656 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257c747-80c2-46a5-a5e9-698b1b3e6dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:02.138063 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257c747-80c2-46a5-a5e9-698b1b3e6dae admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:02.138857 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257c747-80c2-46a5-a5e9-698b1b3e6dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:02.139485 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257c747-80c2-46a5-a5e9-698b1b3e6dae admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:02.146064 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257c747-80c2-46a5-a5e9-698b1b3e6dae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:02.148376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257c747-80c2-46a5-a5e9-698b1b3e6dae admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:02.211660 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1276/2551] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:01 2022] POST /identity/v3/users => generated 282 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:02.211660 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257c747-80c2-46a5-a5e9-698b1b3e6dae admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:02.251764 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257c747-80c2-46a5-a5e9-698b1b3e6dae admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:02.264338 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007da67d-a6cb-4198-a8f3-e8257c5c12b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:02.265567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007da67d-a6cb-4198-a8f3-e8257c5c12b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:02.266149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007da67d-a6cb-4198-a8f3-e8257c5c12b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:02.266712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007da67d-a6cb-4198-a8f3-e8257c5c12b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:02.268901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007da67d-a6cb-4198-a8f3-e8257c5c12b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:02.272298 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007da67d-a6cb-4198-a8f3-e8257c5c12b3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:02.313923 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1277/2552] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:02.316858 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007da67d-a6cb-4198-a8f3-e8257c5c12b3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:02.337461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-209b10c1-0ad9-48ac-870b-6aeef280a052 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b1037d0d805482c871c2f0ca4b00813'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:02.338268 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007da67d-a6cb-4198-a8f3-e8257c5c12b3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:02.348136 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda9fd6-d432-4b3d-8d42-9f967d831152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:02.349507 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda9fd6-d432-4b3d-8d42-9f967d831152 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:02.350347 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda9fd6-d432-4b3d-8d42-9f967d831152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:02.350965 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bda9fd6-d432-4b3d-8d42-9f967d831152 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:02.352865 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bda9fd6-d432-4b3d-8d42-9f967d831152 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:02.355990 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bda9fd6-d432-4b3d-8d42-9f967d831152 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:02.389277 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-209b10c1-0ad9-48ac-870b-6aeef280a052 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b1037d0d805482c871c2f0ca4b00813'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:02.417066 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1278/2553] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:02.418461 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda9fd6-d432-4b3d-8d42-9f967d831152 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:02.422238 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-209b10c1-0ad9-48ac-870b-6aeef280a052 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b1037d0d805482c871c2f0ca4b00813'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:02.423856 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-209b10c1-0ad9-48ac-870b-6aeef280a052 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b1037d0d805482c871c2f0ca4b00813'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:02.456272 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bda9fd6-d432-4b3d-8d42-9f967d831152 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:02.466324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-209b10c1-0ad9-48ac-870b-6aeef280a052 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b1037d0d805482c871c2f0ca4b00813'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:02.489580 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4761ae3-d5e0-45c8-987e-cd334553d6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:02.489580 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4761ae3-d5e0-45c8-987e-cd334553d6ee admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:02.489580 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4761ae3-d5e0-45c8-987e-cd334553d6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:02.489580 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4761ae3-d5e0-45c8-987e-cd334553d6ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:02.490665 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4761ae3-d5e0-45c8-987e-cd334553d6ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:02.490820 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1276/2554] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:01 2022] DELETE /identity/v3/users/7b1037d0d805482c871c2f0ca4b00813 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:02.490820 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b10c1-0ad9-48ac-870b-6aeef280a052 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:02.498385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4761ae3-d5e0-45c8-987e-cd334553d6ee admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:02.499202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-209b10c1-0ad9-48ac-870b-6aeef280a052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:02.523441 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1279/2555] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:02.535641 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4761ae3-d5e0-45c8-987e-cd334553d6ee admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:02.540795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d4761ae3-d5e0-45c8-987e-cd334553d6ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:02.605936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4761ae3-d5e0-45c8-987e-cd334553d6ee admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:02.666736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209b10c1-0ad9-48ac-870b-6aeef280a052 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:02.672366 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd0a82-7d2e-4e49-8607-fb6c538e8b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:02.672997 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd0a82-7d2e-4e49-8607-fb6c538e8b6e admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:02.673280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd0a82-7d2e-4e49-8607-fb6c538e8b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:02.673610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdd0a82-7d2e-4e49-8607-fb6c538e8b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:02.674722 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd0a82-7d2e-4e49-8607-fb6c538e8b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:02.676456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdd0a82-7d2e-4e49-8607-fb6c538e8b6e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:02.688215 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1277/2556] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:02.693876 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd0a82-7d2e-4e49-8607-fb6c538e8b6e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:02.709511 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdd0a82-7d2e-4e49-8607-fb6c538e8b6e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:02.715762 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce13dfa-55ac-4237-8c1d-9c5fed9297a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:02.716458 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce13dfa-55ac-4237-8c1d-9c5fed9297a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:02.716780 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce13dfa-55ac-4237-8c1d-9c5fed9297a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:02.717057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce13dfa-55ac-4237-8c1d-9c5fed9297a8 admin admin] PATH_INFO: `/v3/projects/39ffbd016b784495af3b1be258c5ff6f/users/b32e87e5d8234756bfb63e60854331d2/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:02.719234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a02d3e-914a-44a7-a6ae-8a66f4daf8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:02.721054 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a02d3e-914a-44a7-a6ae-8a66f4daf8ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:02.722472 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a02d3e-914a-44a7-a6ae-8a66f4daf8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:02.722870 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a02d3e-914a-44a7-a6ae-8a66f4daf8ad admin admin] PATH_INFO: `/v3/projects/90512a60a8a34881899ce38456466fdb/users/86b7ebd1f15f40cdaea6d4f1711fd166/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:02.732990 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:02.738525 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce13dfa-55ac-4237-8c1d-9c5fed9297a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39ffbd016b784495af3b1be258c5ff6f, user_id=b32e87e5d8234756bfb63e60854331d2, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:02.742891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce13dfa-55ac-4237-8c1d-9c5fed9297a8 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:02.751029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a02d3e-914a-44a7-a6ae-8a66f4daf8ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=90512a60a8a34881899ce38456466fdb, user_id=86b7ebd1f15f40cdaea6d4f1711fd166, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:02.767620 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a02d3e-914a-44a7-a6ae-8a66f4daf8ad admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:02.779549 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1278/2557] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:02 2022] PUT /identity/v3/projects/39ffbd016b784495af3b1be258c5ff6f/users/b32e87e5d8234756bfb63e60854331d2/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:02.786143 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce13dfa-55ac-4237-8c1d-9c5fed9297a8 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:02.790801 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bce13dfa-55ac-4237-8c1d-9c5fed9297a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:02.792990 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1280/2558] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:02 2022] PUT /identity/v3/projects/90512a60a8a34881899ce38456466fdb/users/86b7ebd1f15f40cdaea6d4f1711fd166/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:02.795205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66917b-bfbf-44c0-bac8-519528e33f3a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:02.795557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66917b-bfbf-44c0-bac8-519528e33f3a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:02.795910 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f66917b-bfbf-44c0-bac8-519528e33f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:02.835609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7f66917b-bfbf-44c0-bac8-519528e33f3a None None] MFA Rules not processed for user `b32e87e5d8234756bfb63e60854331d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:02.837116 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7f66917b-bfbf-44c0-bac8-519528e33f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:02.842200 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:02.896146 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce13dfa-55ac-4237-8c1d-9c5fed9297a8 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:02.920367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1281/2559] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:02 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:02.920367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8da78-6c27-473c-8e03-b7a25878004b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:02.920367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8da78-6c27-473c-8e03-b7a25878004b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:02.920367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d8da78-6c27-473c-8e03-b7a25878004b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:02.957003 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-24d8da78-6c27-473c-8e03-b7a25878004b None None] MFA Rules not processed for user `86b7ebd1f15f40cdaea6d4f1711fd166`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:02.958268 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-24d8da78-6c27-473c-8e03-b7a25878004b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:02.984880 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e935cd-888a-43ff-9610-1484f478c215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:02.985470 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e935cd-888a-43ff-9610-1484f478c215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:02.985783 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e935cd-888a-43ff-9610-1484f478c215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:02.986059 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e935cd-888a-43ff-9610-1484f478c215 admin admin] PATH_INFO: `/v3/projects/344f10a169954071b3023faad8b1c456` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:02.988081 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e935cd-888a-43ff-9610-1484f478c215 admin admin] RBAC: Authorizing `identity:delete_project(project_id=344f10a169954071b3023faad8b1c456)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:02.990069 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e935cd-888a-43ff-9610-1484f478c215 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:03.035618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1282/2560] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:03.036567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8da78-6c27-473c-8e03-b7a25878004b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:03.038044 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-24d8da78-6c27-473c-8e03-b7a25878004b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:03.050713 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-73e935cd-888a-43ff-9610-1484f478c215 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '344f10a169954071b3023faad8b1c456', 'request_id': 'req-73e935cd-888a-43ff-9610-1484f478c215'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:03.061414 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-73e935cd-888a-43ff-9610-1484f478c215 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '344f10a169954071b3023faad8b1c456', 'request_id': 'req-73e935cd-888a-43ff-9610-1484f478c215'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:03.069581 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-73e935cd-888a-43ff-9610-1484f478c215 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '344f10a169954071b3023faad8b1c456', 'request_id': 'req-73e935cd-888a-43ff-9610-1484f478c215'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:03.084301 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-73e935cd-888a-43ff-9610-1484f478c215 admin admin] The token cache is being invalidate because project 344f10a169954071b3023faad8b1c456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:03.084596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-73e935cd-888a-43ff-9610-1484f478c215 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:03.085826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-73e935cd-888a-43ff-9610-1484f478c215 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:03.089628 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1279/2561] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:02 2022] DELETE /identity/v3/projects/344f10a169954071b3023faad8b1c456 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:03.093906 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e935cd-888a-43ff-9610-1484f478c215 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:03.095323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-73e935cd-888a-43ff-9610-1484f478c215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:03.121735 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d8da78-6c27-473c-8e03-b7a25878004b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:03.132252 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:03.165056 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:03.182284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e935cd-888a-43ff-9610-1484f478c215 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:03.185807 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f76b0-a9d1-4df8-8553-97f90227c981 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fecb440c0874ede9f3cf544cfed93ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:03.186499 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f76b0-a9d1-4df8-8553-97f90227c981 service glance] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:03.186783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f76b0-a9d1-4df8-8553-97f90227c981 service glance] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:03.187055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f76b0-a9d1-4df8-8553-97f90227c981 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:03.187489 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65f1fcb-b5c6-443d-b616-69da95889014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:03.188102 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f1fcb-b5c6-443d-b616-69da95889014 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:03.189571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f1fcb-b5c6-443d-b616-69da95889014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:03.189571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65f1fcb-b5c6-443d-b616-69da95889014 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:03.189794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6a2f76b0-a9d1-4df8-8553-97f90227c981 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:03.190140 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f1fcb-b5c6-443d-b616-69da95889014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:03.191777 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65f1fcb-b5c6-443d-b616-69da95889014 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:03.225607 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1280/2562] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:03 2022] POST /identity/v3/projects => generated 387 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:03.226042 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e8569-bccd-4309-8f9a-836eeaeb981d None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:03.226330 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e8569-bccd-4309-8f9a-836eeaeb981d None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:03.226606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31e8569-bccd-4309-8f9a-836eeaeb981d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:03.247368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d31e8569-bccd-4309-8f9a-836eeaeb981d None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:03.248526 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d31e8569-bccd-4309-8f9a-836eeaeb981d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:03.260711 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f76b0-a9d1-4df8-8553-97f90227c981 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:03.262021 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f76b0-a9d1-4df8-8553-97f90227c981 service glance] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:03.272711 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1283/2563] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:03 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:03.273670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f76b0-a9d1-4df8-8553-97f90227c981 service glance] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:03.285767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f76b0-a9d1-4df8-8553-97f90227c981 service glance] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:03.330436 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1281/2564] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:03.332995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57541d5-c818-4561-9e51-a1ec601c88be None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:03.333293 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57541d5-c818-4561-9e51-a1ec601c88be None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:03.334869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57541d5-c818-4561-9e51-a1ec601c88be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:03.356077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e57541d5-c818-4561-9e51-a1ec601c88be None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:03.357296 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e57541d5-c818-4561-9e51-a1ec601c88be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:03.366421 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1282/2565] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:03.371449 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57541d5-c818-4561-9e51-a1ec601c88be None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:03.380541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1f5e9-6d94-4216-a3f2-ec282469e372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:03.381296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1f5e9-6d94-4216-a3f2-ec282469e372 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:03.381679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1f5e9-6d94-4216-a3f2-ec282469e372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:03.382032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c1f5e9-6d94-4216-a3f2-ec282469e372 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:03.382775 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1f5e9-6d94-4216-a3f2-ec282469e372 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:03.384781 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c1f5e9-6d94-4216-a3f2-ec282469e372 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:03.402920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57541d5-c818-4561-9e51-a1ec601c88be None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:03.411005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a294c-9447-462c-bc4a-9c46450150c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:03.411725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a294c-9447-462c-bc4a-9c46450150c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:03.412009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a294c-9447-462c-bc4a-9c46450150c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:03.412586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a294c-9447-462c-bc4a-9c46450150c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:03.412983 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a294c-9447-462c-bc4a-9c46450150c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:36:03.413290 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a294c-9447-462c-bc4a-9c46450150c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:03.415923 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a294c-9447-462c-bc4a-9c46450150c3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:03.423215 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1284/2566] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:03 2022] POST /identity/v3/users => generated 300 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:03.424415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1f5e9-6d94-4216-a3f2-ec282469e372 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:03.431501 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1283/2567] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:36:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:36:03.436867 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c1f5e9-6d94-4216-a3f2-ec282469e372 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:03.438580 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a294c-9447-462c-bc4a-9c46450150c3 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:03.442748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496f237-d7ba-43d4-a88a-7b90182da599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:03.443365 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496f237-d7ba-43d4-a88a-7b90182da599 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:03.443646 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496f237-d7ba-43d4-a88a-7b90182da599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:03.443918 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4496f237-d7ba-43d4-a88a-7b90182da599 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:03.444618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496f237-d7ba-43d4-a88a-7b90182da599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:03.447011 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4496f237-d7ba-43d4-a88a-7b90182da599 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:03.452514 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a294c-9447-462c-bc4a-9c46450150c3 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:03.461563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1285/2568] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:03.462575 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2782d-a1af-4e90-b492-e2a052bc5a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:03.463209 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2782d-a1af-4e90-b492-e2a052bc5a4d admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:03.463489 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2782d-a1af-4e90-b492-e2a052bc5a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:03.463758 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2782d-a1af-4e90-b492-e2a052bc5a4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:03.464421 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2782d-a1af-4e90-b492-e2a052bc5a4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:03.466431 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2782d-a1af-4e90-b492-e2a052bc5a4d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:03.466646 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496f237-d7ba-43d4-a88a-7b90182da599 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:03.478786 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4496f237-d7ba-43d4-a88a-7b90182da599 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:03.491932 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd3982-02b1-4fb3-b828-835331c62a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:03.492897 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd3982-02b1-4fb3-b828-835331c62a9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:03.493188 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd3982-02b1-4fb3-b828-835331c62a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:03.493470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddd3982-02b1-4fb3-b828-835331c62a9d admin admin] PATH_INFO: `/v3/projects/63b609b3a16241a0b21a170f8b5cd97f/users/7744fe1533494091baff2c2dda8abcd9/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:03.504268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1284/2569] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:03 2022] POST /identity/v3/projects => generated 365 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:03.506086 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2782d-a1af-4e90-b492-e2a052bc5a4d admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:03.524266 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd3982-02b1-4fb3-b828-835331c62a9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=63b609b3a16241a0b21a170f8b5cd97f, user_id=7744fe1533494091baff2c2dda8abcd9, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:03.527362 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddd3982-02b1-4fb3-b828-835331c62a9d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:03.549580 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1286/2570] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:03 2022] PUT /identity/v3/projects/63b609b3a16241a0b21a170f8b5cd97f/users/7744fe1533494091baff2c2dda8abcd9/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:03.552539 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd3982-02b1-4fb3-b828-835331c62a9d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:03.608558 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2782d-a1af-4e90-b492-e2a052bc5a4d admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:03.613757 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:03.633694 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:03.665105 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddd3982-02b1-4fb3-b828-835331c62a9d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:03.670343 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ce272-fa6a-455e-aa23-03f9748c9019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:03.671179 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ce272-fa6a-455e-aa23-03f9748c9019 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:03.671632 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ce272-fa6a-455e-aa23-03f9748c9019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:03.672069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ce272-fa6a-455e-aa23-03f9748c9019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:03.673569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ce272-fa6a-455e-aa23-03f9748c9019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:03.675348 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ce272-fa6a-455e-aa23-03f9748c9019 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:03.692717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb50e2-00d9-4e7a-ac20-ade418d17593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:03.693845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb50e2-00d9-4e7a-ac20-ade418d17593 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:03.694134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb50e2-00d9-4e7a-ac20-ade418d17593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:03.694503 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb50e2-00d9-4e7a-ac20-ade418d17593 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:03.695344 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb50e2-00d9-4e7a-ac20-ade418d17593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:03.697884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb50e2-00d9-4e7a-ac20-ade418d17593 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:03.702985 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1287/2571] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:03.706005 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ce272-fa6a-455e-aa23-03f9748c9019 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:03.727608 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ce272-fa6a-455e-aa23-03f9748c9019 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:03.745271 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1285/2572] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:03 2022] POST /identity/v3/users => generated 283 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:03.745486 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e286c-3ebc-4197-be9c-72138b32130b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:03.745486 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e286c-3ebc-4197-be9c-72138b32130b admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:03.745486 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e286c-3ebc-4197-be9c-72138b32130b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:03.745486 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4e286c-3ebc-4197-be9c-72138b32130b admin admin] PATH_INFO: `/v3/projects/63b609b3a16241a0b21a170f8b5cd97f/users/7744fe1533494091baff2c2dda8abcd9/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:03.746363 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb50e2-00d9-4e7a-ac20-ade418d17593 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:03.754485 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e286c-3ebc-4197-be9c-72138b32130b admin admin] RBAC: Authorizing `identity:create_grant(project_id=63b609b3a16241a0b21a170f8b5cd97f, user_id=7744fe1533494091baff2c2dda8abcd9, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:03.758179 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4e286c-3ebc-4197-be9c-72138b32130b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:03.765967 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb50e2-00d9-4e7a-ac20-ade418d17593 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:03.776093 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e512abd9-9742-442f-a2eb-d859119a01c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:03.776728 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512abd9-9742-442f-a2eb-d859119a01c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:03.776998 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512abd9-9742-442f-a2eb-d859119a01c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:03.777267 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512abd9-9742-442f-a2eb-d859119a01c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:03.777998 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512abd9-9742-442f-a2eb-d859119a01c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:03.779669 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e512abd9-9742-442f-a2eb-d859119a01c0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:03.797328 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1288/2573] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:03 2022] PUT /identity/v3/projects/63b609b3a16241a0b21a170f8b5cd97f/users/7744fe1533494091baff2c2dda8abcd9/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:03.798266 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1286/2574] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:03.798364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e286c-3ebc-4197-be9c-72138b32130b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:03.800278 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b7de1-da4a-4637-8191-bff9dfe6628b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:03.800278 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b7de1-da4a-4637-8191-bff9dfe6628b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:03.800468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36b7de1-da4a-4637-8191-bff9dfe6628b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:03.873688 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-d36b7de1-da4a-4637-8191-bff9dfe6628b None None] MFA Rules not processed for user `7744fe1533494091baff2c2dda8abcd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:03.875054 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d36b7de1-da4a-4637-8191-bff9dfe6628b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:04.009356 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4e286c-3ebc-4197-be9c-72138b32130b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:04.040700 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d13ab3-f5c7-41c6-b4ef-0ae669c7eb22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:04.041345 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d13ab3-f5c7-41c6-b4ef-0ae669c7eb22 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:04.041646 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d13ab3-f5c7-41c6-b4ef-0ae669c7eb22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:04.041924 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d13ab3-f5c7-41c6-b4ef-0ae669c7eb22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:04.042634 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d13ab3-f5c7-41c6-b4ef-0ae669c7eb22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:04.055946 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d13ab3-f5c7-41c6-b4ef-0ae669c7eb22 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:04.080262 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1289/2575] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:03 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:04.085070 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d13ab3-f5c7-41c6-b4ef-0ae669c7eb22 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:04.101717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d13ab3-f5c7-41c6-b4ef-0ae669c7eb22 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:04.113257 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcca1f-0029-4f67-99b4-7811598e82ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:04.114232 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcca1f-0029-4f67-99b4-7811598e82ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:04.114845 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcca1f-0029-4f67-99b4-7811598e82ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:04.115306 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcca1f-0029-4f67-99b4-7811598e82ca admin admin] PATH_INFO: `/v3/projects/bcfa576ccd164a80903b4c5c5a791f2c/users/335184fe44ad45bfb5c92c6b7456098b/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:04.133515 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1287/2576] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:03 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:04.151117 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcca1f-0029-4f67-99b4-7811598e82ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcfa576ccd164a80903b4c5c5a791f2c, user_id=335184fe44ad45bfb5c92c6b7456098b, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:04.152258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b7de1-da4a-4637-8191-bff9dfe6628b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:04.154181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d36b7de1-da4a-4637-8191-bff9dfe6628b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:04.154922 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcca1f-0029-4f67-99b4-7811598e82ca admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:04.196171 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1290/2577] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:04 2022] PUT /identity/v3/projects/bcfa576ccd164a80903b4c5c5a791f2c/users/335184fe44ad45bfb5c92c6b7456098b/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:04.206786 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8217d6-da7c-4754-8f42-a6e2498f6d38 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:04.207078 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8217d6-da7c-4754-8f42-a6e2498f6d38 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:04.207357 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8217d6-da7c-4754-8f42-a6e2498f6d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:04.241462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ec8217d6-da7c-4754-8f42-a6e2498f6d38 None None] MFA Rules not processed for user `335184fe44ad45bfb5c92c6b7456098b`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:04.242800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ec8217d6-da7c-4754-8f42-a6e2498f6d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:04.245664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36b7de1-da4a-4637-8191-bff9dfe6628b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:04.313961 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:04.333727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efd3eb-5732-477f-b451-21dbcbc7871c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:04.335670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efd3eb-5732-477f-b451-21dbcbc7871c service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:04.335963 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efd3eb-5732-477f-b451-21dbcbc7871c service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:04.336241 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80efd3eb-5732-477f-b451-21dbcbc7871c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:04.378773 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1291/2578] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:04 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:36:04.383147 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8217d6-da7c-4754-8f42-a6e2498f6d38 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:04.426115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80efd3eb-5732-477f-b451-21dbcbc7871c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:04.427680 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80efd3eb-5732-477f-b451-21dbcbc7871c service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:04.441641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1288/2579] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:04.459671 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:04.478553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efd3eb-5732-477f-b451-21dbcbc7871c service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:04.538901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8217d6-da7c-4754-8f42-a6e2498f6d38 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:04.559860 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df44fb-bef2-4679-b391-96d3d7d35714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:04.560686 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df44fb-bef2-4679-b391-96d3d7d35714 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:04.561072 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df44fb-bef2-4679-b391-96d3d7d35714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:04.561447 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df44fb-bef2-4679-b391-96d3d7d35714 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:04.562216 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df44fb-bef2-4679-b391-96d3d7d35714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:04.564582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9df44fb-bef2-4679-b391-96d3d7d35714 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:04.599017 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80efd3eb-5732-477f-b451-21dbcbc7871c service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:04.610292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146603c8-886e-415d-8196-5f6ab405ceec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:04.611038 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146603c8-886e-415d-8196-5f6ab405ceec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:04.611265 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146603c8-886e-415d-8196-5f6ab405ceec admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:04.611544 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146603c8-886e-415d-8196-5f6ab405ceec admin admin] PATH_INFO: `/v3/users/86b7ebd1f15f40cdaea6d4f1711fd166` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:04.615704 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146603c8-886e-415d-8196-5f6ab405ceec admin admin] RBAC: Authorizing `identity:delete_user(user_id=86b7ebd1f15f40cdaea6d4f1711fd166)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:04.621764 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146603c8-886e-415d-8196-5f6ab405ceec admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:04.630274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1292/2580] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:04 2022] POST /identity/v3/projects => generated 365 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:04.635361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df44fb-bef2-4679-b391-96d3d7d35714 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:04.697765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-146603c8-886e-415d-8196-5f6ab405ceec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86b7ebd1f15f40cdaea6d4f1711fd166'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:04.712432 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-146603c8-886e-415d-8196-5f6ab405ceec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86b7ebd1f15f40cdaea6d4f1711fd166'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:04.723777 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-146603c8-886e-415d-8196-5f6ab405ceec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86b7ebd1f15f40cdaea6d4f1711fd166'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:04.725109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-146603c8-886e-415d-8196-5f6ab405ceec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86b7ebd1f15f40cdaea6d4f1711fd166'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:04.749607 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-146603c8-886e-415d-8196-5f6ab405ceec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86b7ebd1f15f40cdaea6d4f1711fd166'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:04.759192 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1289/2581] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:04 2022] DELETE /identity/v3/users/86b7ebd1f15f40cdaea6d4f1711fd166 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:04.763423 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9df44fb-bef2-4679-b391-96d3d7d35714 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:04.764701 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04f0ee-144e-4036-b027-c5565c1e4a67 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:04.764701 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04f0ee-144e-4036-b027-c5565c1e4a67 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:04.764945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04f0ee-144e-4036-b027-c5565c1e4a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:04.793931 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-db04f0ee-144e-4036-b027-c5565c1e4a67 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:04.795355 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-db04f0ee-144e-4036-b027-c5565c1e4a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:04.801248 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:04.851350 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d6980-5bcf-4f42-b486-e6460699d668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:04.852575 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d6980-5bcf-4f42-b486-e6460699d668 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:04.852964 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d6980-5bcf-4f42-b486-e6460699d668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:04.853918 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26d6980-5bcf-4f42-b486-e6460699d668 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:04.856079 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d6980-5bcf-4f42-b486-e6460699d668 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:04.859570 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26d6980-5bcf-4f42-b486-e6460699d668 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:04.892644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1293/2582] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:04 2022] POST /identity/v3/users => generated 289 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:36:04.897372 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d6980-5bcf-4f42-b486-e6460699d668 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:04.900132 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a26d6980-5bcf-4f42-b486-e6460699d668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:04.911801 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1290/2583] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:04.922121 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db04f0ee-144e-4036-b027-c5565c1e4a67 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:04.924035 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-db04f0ee-144e-4036-b027-c5565c1e4a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:04.927403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26d6980-5bcf-4f42-b486-e6460699d668 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:04.934325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73ed55-e005-4975-9b2f-9b526f45f01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:04.935230 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73ed55-e005-4975-9b2f-9b526f45f01b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:04.935722 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73ed55-e005-4975-9b2f-9b526f45f01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:04.936151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc73ed55-e005-4975-9b2f-9b526f45f01b admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:04.937042 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73ed55-e005-4975-9b2f-9b526f45f01b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:04.939181 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc73ed55-e005-4975-9b2f-9b526f45f01b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:04.954022 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1294/2584] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:04 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:04.959008 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73ed55-e005-4975-9b2f-9b526f45f01b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:04.973200 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc73ed55-e005-4975-9b2f-9b526f45f01b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:04.992410 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db4940-bf30-4be1-b3fd-9b763a60c32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:04.993406 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db4940-bf30-4be1-b3fd-9b763a60c32d admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:04.994931 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db4940-bf30-4be1-b3fd-9b763a60c32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:04.995532 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db4940-bf30-4be1-b3fd-9b763a60c32d admin admin] PATH_INFO: `/v3/projects/c176d1ef989c412d801ace6f033426e4/users/dde1b808cffe4407939a18952a9a4a15/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:05.009335 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db04f0ee-144e-4036-b027-c5565c1e4a67 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:05.013726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7693f96-4a6c-4528-9ded-ea7af53d40af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:05.014982 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7693f96-4a6c-4528-9ded-ea7af53d40af service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:05.015346 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7693f96-4a6c-4528-9ded-ea7af53d40af service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:05.015700 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7693f96-4a6c-4528-9ded-ea7af53d40af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:05.026894 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7693f96-4a6c-4528-9ded-ea7af53d40af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:05.027848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db4940-bf30-4be1-b3fd-9b763a60c32d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c176d1ef989c412d801ace6f033426e4, user_id=dde1b808cffe4407939a18952a9a4a15, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:05.031627 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db4940-bf30-4be1-b3fd-9b763a60c32d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:05.032643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7693f96-4a6c-4528-9ded-ea7af53d40af service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:05.045968 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1291/2585] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:05.071527 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7693f96-4a6c-4528-9ded-ea7af53d40af service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:05.075815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1295/2586] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:04 2022] PUT /identity/v3/projects/c176d1ef989c412d801ace6f033426e4/users/dde1b808cffe4407939a18952a9a4a15/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:05.089902 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7693f96-4a6c-4528-9ded-ea7af53d40af service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:05.276303 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e10fda-7e0b-425c-aa8c-8b2fbe9385bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:05.276303 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e10fda-7e0b-425c-aa8c-8b2fbe9385bc admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:05.276303 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e10fda-7e0b-425c-aa8c-8b2fbe9385bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:05.276303 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e10fda-7e0b-425c-aa8c-8b2fbe9385bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:05.276808 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e10fda-7e0b-425c-aa8c-8b2fbe9385bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:05.282166 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e10fda-7e0b-425c-aa8c-8b2fbe9385bc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:05.318247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1292/2587] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:05.320170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db4940-bf30-4be1-b3fd-9b763a60c32d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:05.351093 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db4940-bf30-4be1-b3fd-9b763a60c32d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:05.362590 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811929f4-2a5d-429e-b081-e330679b1850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:05.364010 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811929f4-2a5d-429e-b081-e330679b1850 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:05.364720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811929f4-2a5d-429e-b081-e330679b1850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:05.365156 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811929f4-2a5d-429e-b081-e330679b1850 admin admin] PATH_INFO: `/v3/projects/c176d1ef989c412d801ace6f033426e4/users/dde1b808cffe4407939a18952a9a4a15/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:05.368942 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811929f4-2a5d-429e-b081-e330679b1850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c176d1ef989c412d801ace6f033426e4, user_id=dde1b808cffe4407939a18952a9a4a15, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:05.376414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811929f4-2a5d-429e-b081-e330679b1850 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:05.418423 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6de4d-c194-4102-96ce-42be201936bb None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:05.418714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6de4d-c194-4102-96ce-42be201936bb None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:05.419069 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c6de4d-c194-4102-96ce-42be201936bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:05.419198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1296/2588] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:05 2022] PUT /identity/v3/projects/c176d1ef989c412d801ace6f033426e4/users/dde1b808cffe4407939a18952a9a4a15/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:05.484866 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-31c6de4d-c194-4102-96ce-42be201936bb None None] MFA Rules not processed for user `dde1b808cffe4407939a18952a9a4a15`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:05.486234 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-31c6de4d-c194-4102-96ce-42be201936bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:05.591224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:05.643664 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1293/2589] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:05 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:05.655790 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811929f4-2a5d-429e-b081-e330679b1850 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:05.658287 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-811929f4-2a5d-429e-b081-e330679b1850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:05.670397 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:05.751284 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811929f4-2a5d-429e-b081-e330679b1850 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:05.757310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686ccf3-bb6e-4a3b-b525-b6bb7038cdc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:05.757310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686ccf3-bb6e-4a3b-b525-b6bb7038cdc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:05.757310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686ccf3-bb6e-4a3b-b525-b6bb7038cdc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:05.757310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686ccf3-bb6e-4a3b-b525-b6bb7038cdc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:05.768973 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686ccf3-bb6e-4a3b-b525-b6bb7038cdc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:05.770133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686ccf3-bb6e-4a3b-b525-b6bb7038cdc5 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:05.781796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1297/2590] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:05 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:06.085745 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6de4d-c194-4102-96ce-42be201936bb None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:06.089117 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-31c6de4d-c194-4102-96ce-42be201936bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:06.248056 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c6de4d-c194-4102-96ce-42be201936bb None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:06.261015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee6f54-f33b-4877-9482-e27d52a3e1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:06.263490 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee6f54-f33b-4877-9482-e27d52a3e1a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:06.263490 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee6f54-f33b-4877-9482-e27d52a3e1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:06.265353 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee6f54-f33b-4877-9482-e27d52a3e1a6 admin admin] PATH_INFO: `/v3/projects/90512a60a8a34881899ce38456466fdb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:06.268575 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee6f54-f33b-4877-9482-e27d52a3e1a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90512a60a8a34881899ce38456466fdb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:06.271033 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee6f54-f33b-4877-9482-e27d52a3e1a6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:06.354865 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f6ee6f54-f33b-4877-9482-e27d52a3e1a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90512a60a8a34881899ce38456466fdb', 'request_id': 'req-f6ee6f54-f33b-4877-9482-e27d52a3e1a6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:06.370419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f6ee6f54-f33b-4877-9482-e27d52a3e1a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90512a60a8a34881899ce38456466fdb', 'request_id': 'req-f6ee6f54-f33b-4877-9482-e27d52a3e1a6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:06.391996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f6ee6f54-f33b-4877-9482-e27d52a3e1a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90512a60a8a34881899ce38456466fdb', 'request_id': 'req-f6ee6f54-f33b-4877-9482-e27d52a3e1a6'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:06.414553 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686ccf3-bb6e-4a3b-b525-b6bb7038cdc5 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:06.421294 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f6ee6f54-f33b-4877-9482-e27d52a3e1a6 admin admin] The token cache is being invalidate because project 90512a60a8a34881899ce38456466fdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:06.421675 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f6ee6f54-f33b-4877-9482-e27d52a3e1a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:06.422354 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-f6ee6f54-f33b-4877-9482-e27d52a3e1a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:06.427766 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1294/2591] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:06 2022] DELETE /identity/v3/projects/90512a60a8a34881899ce38456466fdb => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:06.435369 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee6f54-f33b-4877-9482-e27d52a3e1a6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:06.437106 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686ccf3-bb6e-4a3b-b525-b6bb7038cdc5 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:06.437247 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f6ee6f54-f33b-4877-9482-e27d52a3e1a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:06.545403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01f41c-7f35-4e37-8a03-c092e90c10c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:06.546100 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01f41c-7f35-4e37-8a03-c092e90c10c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:06.546378 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01f41c-7f35-4e37-8a03-c092e90c10c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:06.546649 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01f41c-7f35-4e37-8a03-c092e90c10c4 admin admin] PATH_INFO: `/v3/users/335184fe44ad45bfb5c92c6b7456098b` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:06.547832 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01f41c-7f35-4e37-8a03-c092e90c10c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=335184fe44ad45bfb5c92c6b7456098b)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:06.550827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de01f41c-7f35-4e37-8a03-c092e90c10c4 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:06.579398 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee6f54-f33b-4877-9482-e27d52a3e1a6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:06.590006 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db0541-7528-4041-be46-81ccc6ade65a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:06.590724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db0541-7528-4041-be46-81ccc6ade65a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:06.591005 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db0541-7528-4041-be46-81ccc6ade65a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:06.591282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db0541-7528-4041-be46-81ccc6ade65a admin admin] PATH_INFO: `/v3/users/7744fe1533494091baff2c2dda8abcd9` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:06.592586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db0541-7528-4041-be46-81ccc6ade65a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7744fe1533494091baff2c2dda8abcd9)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:06.594581 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db0541-7528-4041-be46-81ccc6ade65a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:06.617704 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:06.645160 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-de01f41c-7f35-4e37-8a03-c092e90c10c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '335184fe44ad45bfb5c92c6b7456098b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:06.664184 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-de01f41c-7f35-4e37-8a03-c092e90c10c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '335184fe44ad45bfb5c92c6b7456098b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:06.665129 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-de01f41c-7f35-4e37-8a03-c092e90c10c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '335184fe44ad45bfb5c92c6b7456098b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:06.666145 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-de01f41c-7f35-4e37-8a03-c092e90c10c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '335184fe44ad45bfb5c92c6b7456098b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:06.718826 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-de01f41c-7f35-4e37-8a03-c092e90c10c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '335184fe44ad45bfb5c92c6b7456098b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:06.726188 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-19db0541-7528-4041-be46-81ccc6ade65a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7744fe1533494091baff2c2dda8abcd9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:06.756379 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1298/2592] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:06 2022] DELETE /identity/v3/users/335184fe44ad45bfb5c92c6b7456098b => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:06.761349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-19db0541-7528-4041-be46-81ccc6ade65a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7744fe1533494091baff2c2dda8abcd9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:06.761548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892df38-2b5e-4166-9c09-26eaa0502e0b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:06.761732 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892df38-2b5e-4166-9c09-26eaa0502e0b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:06.762824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892df38-2b5e-4166-9c09-26eaa0502e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:06.770904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-19db0541-7528-4041-be46-81ccc6ade65a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7744fe1533494091baff2c2dda8abcd9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:06.771643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-19db0541-7528-4041-be46-81ccc6ade65a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7744fe1533494091baff2c2dda8abcd9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:06.786022 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-19db0541-7528-4041-be46-81ccc6ade65a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7744fe1533494091baff2c2dda8abcd9'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:06.793089 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1295/2593] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:06 2022] DELETE /identity/v3/users/7744fe1533494091baff2c2dda8abcd9 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:06.802050 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0892df38-2b5e-4166-9c09-26eaa0502e0b None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:06.811326 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0892df38-2b5e-4166-9c09-26eaa0502e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:06.853012 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:06.945587 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1299/2594] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:06.970819 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db0541-7528-4041-be46-81ccc6ade65a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:06.973070 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-19db0541-7528-4041-be46-81ccc6ade65a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:07.107256 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db0541-7528-4041-be46-81ccc6ade65a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:07.117078 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015d212-29ec-45c9-8d01-4a5035b52112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:07.117768 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015d212-29ec-45c9-8d01-4a5035b52112 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:07.118058 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015d212-29ec-45c9-8d01-4a5035b52112 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:07.118336 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015d212-29ec-45c9-8d01-4a5035b52112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:07.137872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015d212-29ec-45c9-8d01-4a5035b52112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:07.139713 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015d212-29ec-45c9-8d01-4a5035b52112 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:07.177542 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1296/2595] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:07.522930 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892df38-2b5e-4166-9c09-26eaa0502e0b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:07.522930 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0892df38-2b5e-4166-9c09-26eaa0502e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:07.541924 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892df38-2b5e-4166-9c09-26eaa0502e0b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:07.557915 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3ff928-2c9c-4fdf-bc14-78359873b8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:07.557915 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ff928-2c9c-4fdf-bc14-78359873b8bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:07.557915 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ff928-2c9c-4fdf-bc14-78359873b8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:07.559139 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3ff928-2c9c-4fdf-bc14-78359873b8bf admin admin] PATH_INFO: `/v3/projects/63b609b3a16241a0b21a170f8b5cd97f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:07.559569 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ff928-2c9c-4fdf-bc14-78359873b8bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=63b609b3a16241a0b21a170f8b5cd97f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:07.567285 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3ff928-2c9c-4fdf-bc14-78359873b8bf admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:07.684559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-be3ff928-2c9c-4fdf-bc14-78359873b8bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63b609b3a16241a0b21a170f8b5cd97f', 'request_id': 'req-be3ff928-2c9c-4fdf-bc14-78359873b8bf'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:07.697490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-be3ff928-2c9c-4fdf-bc14-78359873b8bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63b609b3a16241a0b21a170f8b5cd97f', 'request_id': 'req-be3ff928-2c9c-4fdf-bc14-78359873b8bf'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:07.723033 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-be3ff928-2c9c-4fdf-bc14-78359873b8bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63b609b3a16241a0b21a170f8b5cd97f', 'request_id': 'req-be3ff928-2c9c-4fdf-bc14-78359873b8bf'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:07.738797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015d212-29ec-45c9-8d01-4a5035b52112 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:07.740352 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-2015d212-29ec-45c9-8d01-4a5035b52112 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:07.761487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-be3ff928-2c9c-4fdf-bc14-78359873b8bf admin admin] The token cache is being invalidate because project 63b609b3a16241a0b21a170f8b5cd97f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:07.761955 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-be3ff928-2c9c-4fdf-bc14-78359873b8bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:07.767051 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-be3ff928-2c9c-4fdf-bc14-78359873b8bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:07.770040 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015d212-29ec-45c9-8d01-4a5035b52112 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:07.774921 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1300/2596] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:07 2022] DELETE /identity/v3/projects/63b609b3a16241a0b21a170f8b5cd97f => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:07.778650 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa142919-7c46-4339-842b-da53865ad38a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:07.779129 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa142919-7c46-4339-842b-da53865ad38a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:07.779947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa142919-7c46-4339-842b-da53865ad38a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:07.816212 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-fa142919-7c46-4339-842b-da53865ad38a None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:07.821317 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fa142919-7c46-4339-842b-da53865ad38a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:07.908311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e71816-1f19-459e-9a4f-ff4a949141be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:07.911955 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e71816-1f19-459e-9a4f-ff4a949141be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:07.912363 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e71816-1f19-459e-9a4f-ff4a949141be admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:07.919097 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e71816-1f19-459e-9a4f-ff4a949141be admin admin] PATH_INFO: `/v3/projects/bcfa576ccd164a80903b4c5c5a791f2c` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:07.921761 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e71816-1f19-459e-9a4f-ff4a949141be admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcfa576ccd164a80903b4c5c5a791f2c)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:07.922480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1301/2597] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:07.933113 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e71816-1f19-459e-9a4f-ff4a949141be admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:07.934001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24144f1-1586-4e10-bf57-16434d40defe None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:07.934508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24144f1-1586-4e10-bf57-16434d40defe None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:07.935708 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24144f1-1586-4e10-bf57-16434d40defe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:07.972652 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f24144f1-1586-4e10-bf57-16434d40defe None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:07.974219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f24144f1-1586-4e10-bf57-16434d40defe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:07.985310 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1302/2598] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:07.990275 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24144f1-1586-4e10-bf57-16434d40defe None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:08.005899 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24144f1-1586-4e10-bf57-16434d40defe None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:08.012212 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:08.014363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e7767-9868-437f-80e9-604c63422d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:08.015160 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e7767-9868-437f-80e9-604c63422d2b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:08.015506 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e7767-9868-437f-80e9-604c63422d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:08.015987 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e7767-9868-437f-80e9-604c63422d2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:08.016898 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e7767-9868-437f-80e9-604c63422d2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:36:08.017398 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e7767-9868-437f-80e9-604c63422d2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:08.020317 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e7767-9868-437f-80e9-604c63422d2b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:08.034914 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1303/2599] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:36:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:36:08.040614 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e7767-9868-437f-80e9-604c63422d2b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:08.051399 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a4e71816-1f19-459e-9a4f-ff4a949141be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcfa576ccd164a80903b4c5c5a791f2c', 'request_id': 'req-a4e71816-1f19-459e-9a4f-ff4a949141be'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:08.078147 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e7767-9868-437f-80e9-604c63422d2b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:08.084246 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a4e71816-1f19-459e-9a4f-ff4a949141be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcfa576ccd164a80903b4c5c5a791f2c', 'request_id': 'req-a4e71816-1f19-459e-9a4f-ff4a949141be'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:08.095208 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a4e71816-1f19-459e-9a4f-ff4a949141be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcfa576ccd164a80903b4c5c5a791f2c', 'request_id': 'req-a4e71816-1f19-459e-9a4f-ff4a949141be'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:08.126809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a4e71816-1f19-459e-9a4f-ff4a949141be admin admin] The token cache is being invalidate because project bcfa576ccd164a80903b4c5c5a791f2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:08.128864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a4e71816-1f19-459e-9a4f-ff4a949141be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:08.129277 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:08.130688 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-a4e71816-1f19-459e-9a4f-ff4a949141be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:08.137424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1297/2600] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:07 2022] DELETE /identity/v3/projects/bcfa576ccd164a80903b4c5c5a791f2c => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:08.138896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e71816-1f19-459e-9a4f-ff4a949141be admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:08.141099 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a4e71816-1f19-459e-9a4f-ff4a949141be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:08.175879 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b38bd-ab48-4c0d-aa65-5fa632623bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:08.176839 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b38bd-ab48-4c0d-aa65-5fa632623bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:08.177286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b38bd-ab48-4c0d-aa65-5fa632623bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:08.177751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9b38bd-ab48-4c0d-aa65-5fa632623bb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:08.178576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b38bd-ab48-4c0d-aa65-5fa632623bb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:08.182485 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9b38bd-ab48-4c0d-aa65-5fa632623bb5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:08.255726 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1304/2601] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:08 2022] POST /identity/v3/projects => generated 379 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:08.256809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b38bd-ab48-4c0d-aa65-5fa632623bb5 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:08.264183 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5a9b38bd-ab48-4c0d-aa65-5fa632623bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:08.280695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e71816-1f19-459e-9a4f-ff4a949141be admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:08.399312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9b38bd-ab48-4c0d-aa65-5fa632623bb5 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:08.405096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ce68d-864a-4e0c-856e-15c806d29966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:08.405791 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ce68d-864a-4e0c-856e-15c806d29966 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:08.406149 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ce68d-864a-4e0c-856e-15c806d29966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:08.406334 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ce68d-864a-4e0c-856e-15c806d29966 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:08.407132 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ce68d-864a-4e0c-856e-15c806d29966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:08.407944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ee5cb-254e-498a-b91d-6d48d4c1d2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:08.410078 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604ce68d-864a-4e0c-856e-15c806d29966 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:08.410236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ee5cb-254e-498a-b91d-6d48d4c1d2e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:08.410601 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ee5cb-254e-498a-b91d-6d48d4c1d2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:08.410941 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44ee5cb-254e-498a-b91d-6d48d4c1d2e0 admin admin] PATH_INFO: `/v3/users/dde1b808cffe4407939a18952a9a4a15` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:08.412454 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ee5cb-254e-498a-b91d-6d48d4c1d2e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dde1b808cffe4407939a18952a9a4a15)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:08.414541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44ee5cb-254e-498a-b91d-6d48d4c1d2e0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:08.435782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1305/2602] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:08 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:08.435782 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ce68d-864a-4e0c-856e-15c806d29966 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:08.437820 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-604ce68d-864a-4e0c-856e-15c806d29966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:08.454308 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ce68d-864a-4e0c-856e-15c806d29966 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:08.461465 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8871e7-9f1a-4f35-933c-6817b4cf5829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:08.462320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8871e7-9f1a-4f35-933c-6817b4cf5829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:08.462430 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8871e7-9f1a-4f35-933c-6817b4cf5829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:08.462676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8871e7-9f1a-4f35-933c-6817b4cf5829 admin admin] PATH_INFO: `/v3/users/866795813bc545b5a9293ca714095679` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:08.463880 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8871e7-9f1a-4f35-933c-6817b4cf5829 admin admin] RBAC: Authorizing `identity:delete_user(user_id=866795813bc545b5a9293ca714095679)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:08.465833 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8871e7-9f1a-4f35-933c-6817b4cf5829 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:08.482488 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e44ee5cb-254e-498a-b91d-6d48d4c1d2e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dde1b808cffe4407939a18952a9a4a15'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:08.493011 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e44ee5cb-254e-498a-b91d-6d48d4c1d2e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dde1b808cffe4407939a18952a9a4a15'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:08.501287 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e44ee5cb-254e-498a-b91d-6d48d4c1d2e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dde1b808cffe4407939a18952a9a4a15'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:08.502346 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e44ee5cb-254e-498a-b91d-6d48d4c1d2e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dde1b808cffe4407939a18952a9a4a15'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:08.504460 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:08.520218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e44ee5cb-254e-498a-b91d-6d48d4c1d2e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dde1b808cffe4407939a18952a9a4a15'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:08.537582 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1298/2603] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:08 2022] DELETE /identity/v3/users/dde1b808cffe4407939a18952a9a4a15 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:08.537582 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ee5cb-254e-498a-b91d-6d48d4c1d2e0 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:08.542101 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e44ee5cb-254e-498a-b91d-6d48d4c1d2e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:08.558228 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7c8871e7-9f1a-4f35-933c-6817b4cf5829 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '866795813bc545b5a9293ca714095679'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:08.568614 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7c8871e7-9f1a-4f35-933c-6817b4cf5829 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '866795813bc545b5a9293ca714095679'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:08.569904 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7c8871e7-9f1a-4f35-933c-6817b4cf5829 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '866795813bc545b5a9293ca714095679'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:08.571617 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7c8871e7-9f1a-4f35-933c-6817b4cf5829 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '866795813bc545b5a9293ca714095679'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:08.586570 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7c8871e7-9f1a-4f35-933c-6817b4cf5829 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '866795813bc545b5a9293ca714095679'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:08.605811 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1306/2604] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:08 2022] DELETE /identity/v3/users/866795813bc545b5a9293ca714095679 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:08.638873 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44ee5cb-254e-498a-b91d-6d48d4c1d2e0 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:08.724472 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de04476-53c4-4ed2-9fa8-b834d455fcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:08.725256 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de04476-53c4-4ed2-9fa8-b834d455fcca admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:08.725504 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de04476-53c4-4ed2-9fa8-b834d455fcca admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:08.726260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de04476-53c4-4ed2-9fa8-b834d455fcca admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:08.726509 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de04476-53c4-4ed2-9fa8-b834d455fcca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:08.728189 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de04476-53c4-4ed2-9fa8-b834d455fcca admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:08.764422 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1299/2605] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:08.769780 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8871e7-9f1a-4f35-933c-6817b4cf5829 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:08.771450 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7c8871e7-9f1a-4f35-933c-6817b4cf5829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:08.788759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8871e7-9f1a-4f35-933c-6817b4cf5829 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:08.793892 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb07e0-31fc-4a45-a9eb-b15bacc405fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:08.794525 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb07e0-31fc-4a45-a9eb-b15bacc405fb admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:08.795755 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb07e0-31fc-4a45-a9eb-b15bacc405fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:08.795755 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb07e0-31fc-4a45-a9eb-b15bacc405fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:08.796110 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb07e0-31fc-4a45-a9eb-b15bacc405fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:08.798510 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb07e0-31fc-4a45-a9eb-b15bacc405fb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:08.814446 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1307/2606] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:08 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:36:08.819304 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de04476-53c4-4ed2-9fa8-b834d455fcca admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:08.830984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de04476-53c4-4ed2-9fa8-b834d455fcca admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:08.836789 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef84dc1b-dc2b-46ea-813e-0455029510b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:08.837774 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84dc1b-dc2b-46ea-813e-0455029510b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:08.841290 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84dc1b-dc2b-46ea-813e-0455029510b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:08.841290 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef84dc1b-dc2b-46ea-813e-0455029510b5 admin admin] PATH_INFO: `/v3/projects/983e5f7586e3435bb6442b41b467ddd1/users/1b0b33e7d4af482183de5099ca60e6a4/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:08.860788 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84dc1b-dc2b-46ea-813e-0455029510b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=983e5f7586e3435bb6442b41b467ddd1, user_id=1b0b33e7d4af482183de5099ca60e6a4, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:08.867834 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef84dc1b-dc2b-46ea-813e-0455029510b5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:08.899374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1300/2607] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:08 2022] PUT /identity/v3/projects/983e5f7586e3435bb6442b41b467ddd1/users/1b0b33e7d4af482183de5099ca60e6a4/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:08.910484 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb07e0-31fc-4a45-a9eb-b15bacc405fb admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:08.911554 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6188188-671a-49ca-b971-1de9fa9c4904 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:08.912163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6188188-671a-49ca-b971-1de9fa9c4904 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:08.912903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-57bb07e0-31fc-4a45-a9eb-b15bacc405fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:08.913767 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6188188-671a-49ca-b971-1de9fa9c4904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:08.925866 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:08.938220 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb07e0-31fc-4a45-a9eb-b15bacc405fb admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:08.962343 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b6188188-671a-49ca-b971-1de9fa9c4904 None None] MFA Rules not processed for user `1b0b33e7d4af482183de5099ca60e6a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:08.963703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b6188188-671a-49ca-b971-1de9fa9c4904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:09.032322 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:09.035103 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9440288-48ab-41c8-934e-4b05d4fbc55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:09.035925 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9440288-48ab-41c8-934e-4b05d4fbc55b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:09.036794 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9440288-48ab-41c8-934e-4b05d4fbc55b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:09.037177 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9440288-48ab-41c8-934e-4b05d4fbc55b admin admin] PATH_INFO: `/v3/projects/2406f01f7b3b4941a0bcd97a68d3d510` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:09.039416 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9440288-48ab-41c8-934e-4b05d4fbc55b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2406f01f7b3b4941a0bcd97a68d3d510)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:09.041619 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9440288-48ab-41c8-934e-4b05d4fbc55b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:09.057429 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1301/2608] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:09.057429 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6188188-671a-49ca-b971-1de9fa9c4904 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:09.057429 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b6188188-671a-49ca-b971-1de9fa9c4904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:09.072207 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6188188-671a-49ca-b971-1de9fa9c4904 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:09.077208 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61481529-f49c-497a-ac38-9d77aed8771e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:09.078002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61481529-f49c-497a-ac38-9d77aed8771e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:09.078375 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61481529-f49c-497a-ac38-9d77aed8771e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:09.078729 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61481529-f49c-497a-ac38-9d77aed8771e admin admin] PATH_INFO: `/v3/projects/c176d1ef989c412d801ace6f033426e4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:09.080001 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61481529-f49c-497a-ac38-9d77aed8771e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c176d1ef989c412d801ace6f033426e4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:09.082804 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61481529-f49c-497a-ac38-9d77aed8771e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:09.123802 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f9440288-48ab-41c8-934e-4b05d4fbc55b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2406f01f7b3b4941a0bcd97a68d3d510', 'request_id': 'req-f9440288-48ab-41c8-934e-4b05d4fbc55b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:09.132001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f9440288-48ab-41c8-934e-4b05d4fbc55b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2406f01f7b3b4941a0bcd97a68d3d510', 'request_id': 'req-f9440288-48ab-41c8-934e-4b05d4fbc55b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:09.138866 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f9440288-48ab-41c8-934e-4b05d4fbc55b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2406f01f7b3b4941a0bcd97a68d3d510', 'request_id': 'req-f9440288-48ab-41c8-934e-4b05d4fbc55b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:09.152392 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-61481529-f49c-497a-ac38-9d77aed8771e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c176d1ef989c412d801ace6f033426e4', 'request_id': 'req-61481529-f49c-497a-ac38-9d77aed8771e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:09.163799 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-61481529-f49c-497a-ac38-9d77aed8771e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c176d1ef989c412d801ace6f033426e4', 'request_id': 'req-61481529-f49c-497a-ac38-9d77aed8771e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:09.164938 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f9440288-48ab-41c8-934e-4b05d4fbc55b admin admin] The token cache is being invalidate because project 2406f01f7b3b4941a0bcd97a68d3d510 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:09.165302 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f9440288-48ab-41c8-934e-4b05d4fbc55b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:09.166035 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f9440288-48ab-41c8-934e-4b05d4fbc55b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:09.168241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1308/2609] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:08 2022] DELETE /identity/v3/projects/2406f01f7b3b4941a0bcd97a68d3d510 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:09.169093 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9440288-48ab-41c8-934e-4b05d4fbc55b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:09.170460 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f9440288-48ab-41c8-934e-4b05d4fbc55b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:09.175145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-61481529-f49c-497a-ac38-9d77aed8771e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c176d1ef989c412d801ace6f033426e4', 'request_id': 'req-61481529-f49c-497a-ac38-9d77aed8771e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:09.191734 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-61481529-f49c-497a-ac38-9d77aed8771e admin admin] The token cache is being invalidate because project c176d1ef989c412d801ace6f033426e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:09.192057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-61481529-f49c-497a-ac38-9d77aed8771e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:09.192713 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-61481529-f49c-497a-ac38-9d77aed8771e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:09.194365 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1302/2610] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:09 2022] DELETE /identity/v3/projects/c176d1ef989c412d801ace6f033426e4 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:09.198346 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61481529-f49c-497a-ac38-9d77aed8771e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:09.199668 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-61481529-f49c-497a-ac38-9d77aed8771e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:09.244784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9440288-48ab-41c8-934e-4b05d4fbc55b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:09.305098 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61481529-f49c-497a-ac38-9d77aed8771e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:09.311377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791ff77-6535-43a3-824b-1339aa21c92e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:09.312445 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791ff77-6535-43a3-824b-1339aa21c92e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:09.312947 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791ff77-6535-43a3-824b-1339aa21c92e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:09.313438 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791ff77-6535-43a3-824b-1339aa21c92e admin admin] PATH_INFO: `/v3/users/a094696b3532489c8b585fed82aca471` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:09.314982 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791ff77-6535-43a3-824b-1339aa21c92e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a094696b3532489c8b585fed82aca471)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:09.317394 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791ff77-6535-43a3-824b-1339aa21c92e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:09.319488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:09.344799 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d331e-0a3c-4c51-a5f3-b13a79b2a89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:09.345541 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d331e-0a3c-4c51-a5f3-b13a79b2a89d admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:09.345841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d331e-0a3c-4c51-a5f3-b13a79b2a89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:09.347611 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d331e-0a3c-4c51-a5f3-b13a79b2a89d admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:09.352414 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d331e-0a3c-4c51-a5f3-b13a79b2a89d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:09.354679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d331e-0a3c-4c51-a5f3-b13a79b2a89d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:09.380266 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b791ff77-6535-43a3-824b-1339aa21c92e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a094696b3532489c8b585fed82aca471'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:09.389663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1309/2611] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:09 2022] POST /identity/v3/projects => generated 383 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:09.389663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a7c00-3d48-4381-ba8e-a25b0dda9c9c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:09.396359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a7c00-3d48-4381-ba8e-a25b0dda9c9c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:09.396734 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a7c00-3d48-4381-ba8e-a25b0dda9c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:09.398046 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b791ff77-6535-43a3-824b-1339aa21c92e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a094696b3532489c8b585fed82aca471'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:09.408270 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b791ff77-6535-43a3-824b-1339aa21c92e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a094696b3532489c8b585fed82aca471'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:09.409324 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b791ff77-6535-43a3-824b-1339aa21c92e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a094696b3532489c8b585fed82aca471'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:09.417926 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b68a7c00-3d48-4381-ba8e-a25b0dda9c9c None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:09.419117 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b68a7c00-3d48-4381-ba8e-a25b0dda9c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:09.425317 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:09.429740 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b791ff77-6535-43a3-824b-1339aa21c92e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a094696b3532489c8b585fed82aca471'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:09.446423 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1303/2612] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:09 2022] DELETE /identity/v3/users/a094696b3532489c8b585fed82aca471 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:09.447616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791ff77-6535-43a3-824b-1339aa21c92e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:09.462315 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b791ff77-6535-43a3-824b-1339aa21c92e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:09.504392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1310/2613] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:09.513695 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0c65d-7881-4cff-a99c-8432f62946f5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:09.513988 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0c65d-7881-4cff-a99c-8432f62946f5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:09.514265 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0c65d-7881-4cff-a99c-8432f62946f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:09.551468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-86b0c65d-7881-4cff-a99c-8432f62946f5 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:09.553299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-86b0c65d-7881-4cff-a99c-8432f62946f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:09.574284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791ff77-6535-43a3-824b-1339aa21c92e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:09.588513 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe286cc-74db-46e5-90cc-362a74009eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:09.589163 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe286cc-74db-46e5-90cc-362a74009eed admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:09.589448 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe286cc-74db-46e5-90cc-362a74009eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:09.591499 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe286cc-74db-46e5-90cc-362a74009eed admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:09.591980 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe286cc-74db-46e5-90cc-362a74009eed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:09.597966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe286cc-74db-46e5-90cc-362a74009eed admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:09.643751 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1304/2614] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:09 2022] POST /identity/v3/users => generated 298 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:09.652702 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe286cc-74db-46e5-90cc-362a74009eed admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:09.667179 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe286cc-74db-46e5-90cc-362a74009eed admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:09.672366 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1311/2615] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:09.673508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0c65d-7881-4cff-a99c-8432f62946f5 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:09.678384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849374aa-bc09-4793-b431-761d71ab72fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:09.679385 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849374aa-bc09-4793-b431-761d71ab72fa admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:09.679950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849374aa-bc09-4793-b431-761d71ab72fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:09.680505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849374aa-bc09-4793-b431-761d71ab72fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:09.681956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849374aa-bc09-4793-b431-761d71ab72fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:09.683733 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849374aa-bc09-4793-b431-761d71ab72fa admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:09.701565 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1305/2616] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:36:09.701718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0c65d-7881-4cff-a99c-8432f62946f5 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:09.703128 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849374aa-bc09-4793-b431-761d71ab72fa admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:09.711913 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639072f-b78f-4b10-be2a-0790aa625076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:09.713094 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639072f-b78f-4b10-be2a-0790aa625076 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:09.713094 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639072f-b78f-4b10-be2a-0790aa625076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:09.713094 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639072f-b78f-4b10-be2a-0790aa625076 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:09.713809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639072f-b78f-4b10-be2a-0790aa625076 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:36:09.714457 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639072f-b78f-4b10-be2a-0790aa625076 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:09.716555 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639072f-b78f-4b10-be2a-0790aa625076 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:09.724501 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849374aa-bc09-4793-b431-761d71ab72fa admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:09.733842 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c043a2-9c8e-4eab-9b4d-ac91e02991a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:09.738856 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1312/2617] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:36:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:36:09.742053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c043a2-9c8e-4eab-9b4d-ac91e02991a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:09.742512 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c043a2-9c8e-4eab-9b4d-ac91e02991a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:09.743092 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c043a2-9c8e-4eab-9b4d-ac91e02991a3 admin admin] PATH_INFO: `/v3/projects/fd6d9a481f8b4c4b9dd9dad199f2160f/users/8b77687caf504842a748f0a4fd9ade54/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:09.743834 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639072f-b78f-4b10-be2a-0790aa625076 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:09.768536 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c043a2-9c8e-4eab-9b4d-ac91e02991a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd6d9a481f8b4c4b9dd9dad199f2160f, user_id=8b77687caf504842a748f0a4fd9ade54, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:09.773186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c043a2-9c8e-4eab-9b4d-ac91e02991a3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:09.778582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639072f-b78f-4b10-be2a-0790aa625076 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:09.785918 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bcc48-f5f7-468b-adfc-00db95b77d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:09.786498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bcc48-f5f7-468b-adfc-00db95b77d8c admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:09.786992 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bcc48-f5f7-468b-adfc-00db95b77d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:09.787161 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59bcc48-f5f7-468b-adfc-00db95b77d8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:09.787826 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59bcc48-f5f7-468b-adfc-00db95b77d8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:09.790103 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59bcc48-f5f7-468b-adfc-00db95b77d8c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:09.802913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1306/2618] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:09 2022] PUT /identity/v3/projects/fd6d9a481f8b4c4b9dd9dad199f2160f/users/8b77687caf504842a748f0a4fd9ade54/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:09.807869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c043a2-9c8e-4eab-9b4d-ac91e02991a3 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:09.851739 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1313/2619] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:09 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:09.851739 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bcc48-f5f7-468b-adfc-00db95b77d8c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:09.853614 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f59bcc48-f5f7-468b-adfc-00db95b77d8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:09.910986 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c043a2-9c8e-4eab-9b4d-ac91e02991a3 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:09.929217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:09.984794 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:09.990860 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59bcc48-f5f7-468b-adfc-00db95b77d8c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:09.996298 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8492f-479f-449f-801e-fb33b4840bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:09.997031 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8492f-479f-449f-801e-fb33b4840bbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:09.997319 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8492f-479f-449f-801e-fb33b4840bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:09.997627 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc8492f-479f-449f-801e-fb33b4840bbc admin admin] PATH_INFO: `/v3/projects/80d3cde38cd9481ab0dc9d3836de3359` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:09.998967 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8492f-479f-449f-801e-fb33b4840bbc admin admin] RBAC: Authorizing `identity:delete_project(project_id=80d3cde38cd9481ab0dc9d3836de3359)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:10.002598 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0617270-121e-4e18-a246-f50236ad4425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:10.003243 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0617270-121e-4e18-a246-f50236ad4425 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:10.006146 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0617270-121e-4e18-a246-f50236ad4425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:10.006443 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc8492f-479f-449f-801e-fb33b4840bbc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:10.006583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0617270-121e-4e18-a246-f50236ad4425 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:10.010430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0617270-121e-4e18-a246-f50236ad4425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:10.012245 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0617270-121e-4e18-a246-f50236ad4425 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:10.026723 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1307/2620] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:09 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:10.027602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0617270-121e-4e18-a246-f50236ad4425 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:10.043755 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0617270-121e-4e18-a246-f50236ad4425 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:10.051765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491fc0f-2ea6-44f6-bedb-0dc69bbc64df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:10.052406 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491fc0f-2ea6-44f6-bedb-0dc69bbc64df admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:10.053286 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491fc0f-2ea6-44f6-bedb-0dc69bbc64df admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:10.053864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8491fc0f-2ea6-44f6-bedb-0dc69bbc64df admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:10.054549 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491fc0f-2ea6-44f6-bedb-0dc69bbc64df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:10.059288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8491fc0f-2ea6-44f6-bedb-0dc69bbc64df admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:10.091097 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1308/2621] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:10 2022] POST /identity/v3/users => generated 294 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:10.095352 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491fc0f-2ea6-44f6-bedb-0dc69bbc64df admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:10.099013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4fc8492f-479f-449f-801e-fb33b4840bbc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80d3cde38cd9481ab0dc9d3836de3359', 'request_id': 'req-4fc8492f-479f-449f-801e-fb33b4840bbc'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:10.113920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8491fc0f-2ea6-44f6-bedb-0dc69bbc64df admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:10.115294 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4fc8492f-479f-449f-801e-fb33b4840bbc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80d3cde38cd9481ab0dc9d3836de3359', 'request_id': 'req-4fc8492f-479f-449f-801e-fb33b4840bbc'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:10.136729 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4fc8492f-479f-449f-801e-fb33b4840bbc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80d3cde38cd9481ab0dc9d3836de3359', 'request_id': 'req-4fc8492f-479f-449f-801e-fb33b4840bbc'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:10.164383 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4fc8492f-479f-449f-801e-fb33b4840bbc admin admin] The token cache is being invalidate because project 80d3cde38cd9481ab0dc9d3836de3359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:10.164383 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4fc8492f-479f-449f-801e-fb33b4840bbc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:10.164904 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4fc8492f-479f-449f-801e-fb33b4840bbc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:10.166495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1314/2622] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:09 2022] DELETE /identity/v3/projects/80d3cde38cd9481ab0dc9d3836de3359 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:10.167267 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8492f-479f-449f-801e-fb33b4840bbc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:10.169049 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4fc8492f-479f-449f-801e-fb33b4840bbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:10.292600 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b064a-6d4c-441f-b542-f95cac295a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:10.297557 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b064a-6d4c-441f-b542-f95cac295a5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:10.297917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b064a-6d4c-441f-b542-f95cac295a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:10.298210 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b064a-6d4c-441f-b542-f95cac295a5b admin admin] PATH_INFO: `/v3/projects/fd6d9a481f8b4c4b9dd9dad199f2160f/users/8b77687caf504842a748f0a4fd9ade54/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:10.302310 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b064a-6d4c-441f-b542-f95cac295a5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd6d9a481f8b4c4b9dd9dad199f2160f, user_id=8b77687caf504842a748f0a4fd9ade54, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:10.313823 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b064a-6d4c-441f-b542-f95cac295a5b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:10.350499 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1309/2623] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:10 2022] PUT /identity/v3/projects/fd6d9a481f8b4c4b9dd9dad199f2160f/users/8b77687caf504842a748f0a4fd9ade54/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:10.365695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee2ee3-718f-4c00-b592-3eda7feabf4b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:10.366072 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee2ee3-718f-4c00-b592-3eda7feabf4b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:10.366903 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee2ee3-718f-4c00-b592-3eda7feabf4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:10.406905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc8492f-479f-449f-801e-fb33b4840bbc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:10.418643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f7ee2ee3-718f-4c00-b592-3eda7feabf4b None None] MFA Rules not processed for user `8b77687caf504842a748f0a4fd9ade54`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:10.420250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f7ee2ee3-718f-4c00-b592-3eda7feabf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:10.525133 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c86bf0-fd62-423e-b67b-decfed8d9268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:10.527009 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c86bf0-fd62-423e-b67b-decfed8d9268 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:10.527300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c86bf0-fd62-423e-b67b-decfed8d9268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:10.527579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c86bf0-fd62-423e-b67b-decfed8d9268 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:10.528305 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c86bf0-fd62-423e-b67b-decfed8d9268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:10.529998 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c86bf0-fd62-423e-b67b-decfed8d9268 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:10.573246 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1315/2624] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:10.579134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c86bf0-fd62-423e-b67b-decfed8d9268 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:10.624610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1310/2625] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:10.629511 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c86bf0-fd62-423e-b67b-decfed8d9268 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:10.637910 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0661a7b-7565-48f4-8be9-526815da13b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:10.642915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee2ee3-718f-4c00-b592-3eda7feabf4b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:10.645088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f7ee2ee3-718f-4c00-b592-3eda7feabf4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:10.646096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0661a7b-7565-48f4-8be9-526815da13b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:10.646392 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0661a7b-7565-48f4-8be9-526815da13b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:10.646674 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0661a7b-7565-48f4-8be9-526815da13b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:10.647420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0661a7b-7565-48f4-8be9-526815da13b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:10.649323 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0661a7b-7565-48f4-8be9-526815da13b5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:10.650855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:10.689293 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1316/2626] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:36:10.691498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0661a7b-7565-48f4-8be9-526815da13b5 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:10.693760 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b0661a7b-7565-48f4-8be9-526815da13b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:10.718044 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee2ee3-718f-4c00-b592-3eda7feabf4b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:10.726323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eaed6e-37ce-4804-81f4-c91d69bb2bd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:10.731122 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaed6e-37ce-4804-81f4-c91d69bb2bd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:10.731419 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaed6e-37ce-4804-81f4-c91d69bb2bd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:10.731701 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eaed6e-37ce-4804-81f4-c91d69bb2bd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:10.744400 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:10.747259 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eaed6e-37ce-4804-81f4-c91d69bb2bd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:10.749028 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eaed6e-37ce-4804-81f4-c91d69bb2bd0 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:10.765044 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1311/2627] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:36:10.766053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eaed6e-37ce-4804-81f4-c91d69bb2bd0 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:10.767531 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d9eaed6e-37ce-4804-81f4-c91d69bb2bd0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:10.781917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eaed6e-37ce-4804-81f4-c91d69bb2bd0 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:10.787199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4221fb16-3106-4949-a2c3-1fdbf5fde4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:10.787946 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221fb16-3106-4949-a2c3-1fdbf5fde4bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:10.788166 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221fb16-3106-4949-a2c3-1fdbf5fde4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:10.788446 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221fb16-3106-4949-a2c3-1fdbf5fde4bf admin admin] PATH_INFO: `/v3/users/21faee3ebc194e709a933b6fbb7640d4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:10.789882 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4221fb16-3106-4949-a2c3-1fdbf5fde4bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=21faee3ebc194e709a933b6fbb7640d4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:10.791669 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4221fb16-3106-4949-a2c3-1fdbf5fde4bf admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:10.838316 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0661a7b-7565-48f4-8be9-526815da13b5 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:10.845320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af28a7-00ed-4b5b-9466-539727c3e6d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe003cd7bfbe4b92b1d64d300c3e7d8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:10.847046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af28a7-00ed-4b5b-9466-539727c3e6d0 service placement] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:10.847343 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af28a7-00ed-4b5b-9466-539727c3e6d0 service placement] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:10.847627 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4af28a7-00ed-4b5b-9466-539727c3e6d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:10.853186 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a4af28a7-00ed-4b5b-9466-539727c3e6d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:10.857468 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4221fb16-3106-4949-a2c3-1fdbf5fde4bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21faee3ebc194e709a933b6fbb7640d4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:10.873558 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4221fb16-3106-4949-a2c3-1fdbf5fde4bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21faee3ebc194e709a933b6fbb7640d4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:10.876115 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af28a7-00ed-4b5b-9466-539727c3e6d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:10.878633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4af28a7-00ed-4b5b-9466-539727c3e6d0 service placement] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:10.886344 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4221fb16-3106-4949-a2c3-1fdbf5fde4bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21faee3ebc194e709a933b6fbb7640d4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:10.887791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4221fb16-3106-4949-a2c3-1fdbf5fde4bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21faee3ebc194e709a933b6fbb7640d4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:10.899420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1317/2628] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:10.900698 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af28a7-00ed-4b5b-9466-539727c3e6d0 service placement] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:10.928851 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4221fb16-3106-4949-a2c3-1fdbf5fde4bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21faee3ebc194e709a933b6fbb7640d4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:10.931994 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4af28a7-00ed-4b5b-9466-539727c3e6d0 service placement] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:10.938863 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed797be3-3146-4d1f-8c1d-a60fe6b1b388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:10.940227 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1312/2629] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:10 2022] DELETE /identity/v3/users/21faee3ebc194e709a933b6fbb7640d4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:10.940621 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed797be3-3146-4d1f-8c1d-a60fe6b1b388 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:10.940621 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed797be3-3146-4d1f-8c1d-a60fe6b1b388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:10.940822 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4221fb16-3106-4949-a2c3-1fdbf5fde4bf admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:10.941670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed797be3-3146-4d1f-8c1d-a60fe6b1b388 admin admin] PATH_INFO: `/v3/projects/a5fd791817be4343bc045308735998bb/users/bc5fb76567e64b13a4482ca6b9dbf996/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:10.941810 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4221fb16-3106-4949-a2c3-1fdbf5fde4bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:10.968495 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed797be3-3146-4d1f-8c1d-a60fe6b1b388 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5fd791817be4343bc045308735998bb, user_id=bc5fb76567e64b13a4482ca6b9dbf996, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:10.974341 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed797be3-3146-4d1f-8c1d-a60fe6b1b388 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:10.996910 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1318/2630] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:10 2022] PUT /identity/v3/projects/a5fd791817be4343bc045308735998bb/users/bc5fb76567e64b13a4482ca6b9dbf996/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:11.008597 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ab527-bc3a-428f-ade3-72d3983f3a6e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:11.008912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ab527-bc3a-428f-ade3-72d3983f3a6e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:11.009228 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75ab527-bc3a-428f-ade3-72d3983f3a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:11.029867 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4221fb16-3106-4949-a2c3-1fdbf5fde4bf admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:11.053210 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b75ab527-bc3a-428f-ade3-72d3983f3a6e None None] MFA Rules not processed for user `bc5fb76567e64b13a4482ca6b9dbf996`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:11.054088 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b75ab527-bc3a-428f-ade3-72d3983f3a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:11.115312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dde8dd-ee8c-45a8-b907-a413550d1a1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:11.115872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dde8dd-ee8c-45a8-b907-a413550d1a1b service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:11.116214 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dde8dd-ee8c-45a8-b907-a413550d1a1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:11.116502 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dde8dd-ee8c-45a8-b907-a413550d1a1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:11.124695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-31dde8dd-ee8c-45a8-b907-a413550d1a1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:11.160092 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1319/2631] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:11.165842 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ab527-bc3a-428f-ade3-72d3983f3a6e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:11.170046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b75ab527-bc3a-428f-ade3-72d3983f3a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:11.193927 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:11.228876 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dde8dd-ee8c-45a8-b907-a413550d1a1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:11.230320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dde8dd-ee8c-45a8-b907-a413550d1a1b service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:11.235536 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:11.247984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1313/2632] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:11.249254 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dde8dd-ee8c-45a8-b907-a413550d1a1b service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:11.251472 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-31dde8dd-ee8c-45a8-b907-a413550d1a1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:11.308217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75ab527-bc3a-428f-ade3-72d3983f3a6e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:11.318974 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61db2d9-bcc9-4bca-b994-54f09f3589df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:11.319926 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61db2d9-bcc9-4bca-b994-54f09f3589df admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:11.320373 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61db2d9-bcc9-4bca-b994-54f09f3589df admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:11.320818 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61db2d9-bcc9-4bca-b994-54f09f3589df admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:11.321981 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61db2d9-bcc9-4bca-b994-54f09f3589df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:11.324256 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61db2d9-bcc9-4bca-b994-54f09f3589df admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:11.338885 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dde8dd-ee8c-45a8-b907-a413550d1a1b service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:11.346051 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f034fc-c8bb-452a-b1ac-f96be9d76840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:11.347902 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f034fc-c8bb-452a-b1ac-f96be9d76840 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:11.348321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f034fc-c8bb-452a-b1ac-f96be9d76840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:11.348694 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f034fc-c8bb-452a-b1ac-f96be9d76840 admin admin] PATH_INFO: `/v3/projects/a4f75425fc254368bd7870a27d8b0ebe` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:11.350992 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f034fc-c8bb-452a-b1ac-f96be9d76840 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4f75425fc254368bd7870a27d8b0ebe)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:11.353639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f034fc-c8bb-452a-b1ac-f96be9d76840 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:11.366335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1320/2633] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:11 2022] POST /identity/v3/projects => generated 385 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:36:11.371458 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61db2d9-bcc9-4bca-b994-54f09f3589df admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:11.422765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-86f034fc-c8bb-452a-b1ac-f96be9d76840 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4f75425fc254368bd7870a27d8b0ebe', 'request_id': 'req-86f034fc-c8bb-452a-b1ac-f96be9d76840'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:11.437390 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-86f034fc-c8bb-452a-b1ac-f96be9d76840 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4f75425fc254368bd7870a27d8b0ebe', 'request_id': 'req-86f034fc-c8bb-452a-b1ac-f96be9d76840'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:11.450228 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-86f034fc-c8bb-452a-b1ac-f96be9d76840 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4f75425fc254368bd7870a27d8b0ebe', 'request_id': 'req-86f034fc-c8bb-452a-b1ac-f96be9d76840'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:11.464683 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61db2d9-bcc9-4bca-b994-54f09f3589df admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:11.472463 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-86f034fc-c8bb-452a-b1ac-f96be9d76840 admin admin] The token cache is being invalidate because project a4f75425fc254368bd7870a27d8b0ebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:11.472463 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-86f034fc-c8bb-452a-b1ac-f96be9d76840 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:11.475722 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-86f034fc-c8bb-452a-b1ac-f96be9d76840 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:11.479118 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1314/2634] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:11 2022] DELETE /identity/v3/projects/a4f75425fc254368bd7870a27d8b0ebe => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:11.483252 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f034fc-c8bb-452a-b1ac-f96be9d76840 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:11.484742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-86f034fc-c8bb-452a-b1ac-f96be9d76840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:11.551248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363ad9f-f5ce-4414-9e9b-9333eda09cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:11.551999 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363ad9f-f5ce-4414-9e9b-9333eda09cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:11.552348 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363ad9f-f5ce-4414-9e9b-9333eda09cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:11.552703 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363ad9f-f5ce-4414-9e9b-9333eda09cd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:11.553448 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9363ad9f-f5ce-4414-9e9b-9333eda09cd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:11.555584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9363ad9f-f5ce-4414-9e9b-9333eda09cd1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:11.588649 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1321/2635] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:11 2022] POST /identity/v3/users => generated 299 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:11.592678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363ad9f-f5ce-4414-9e9b-9333eda09cd1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:11.594979 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f034fc-c8bb-452a-b1ac-f96be9d76840 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:11.595718 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9363ad9f-f5ce-4414-9e9b-9333eda09cd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:11.603409 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc556e-6e82-44e7-98a8-d38a7e5848d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:11.604379 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc556e-6e82-44e7-98a8-d38a7e5848d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:11.605036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc556e-6e82-44e7-98a8-d38a7e5848d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:11.605514 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc556e-6e82-44e7-98a8-d38a7e5848d5 admin admin] PATH_INFO: `/v3/users/91b3a982f4484b6c9e8ef46494f4c92a` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:11.607449 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc556e-6e82-44e7-98a8-d38a7e5848d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91b3a982f4484b6c9e8ef46494f4c92a)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:11.609931 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc556e-6e82-44e7-98a8-d38a7e5848d5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:11.612522 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363ad9f-f5ce-4414-9e9b-9333eda09cd1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:11.626100 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:11.630009 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e2da5-c368-49e9-ac4d-cafe0f67ced7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:11.631253 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e2da5-c368-49e9-ac4d-cafe0f67ced7 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:11.631599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e2da5-c368-49e9-ac4d-cafe0f67ced7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:11.632300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e2da5-c368-49e9-ac4d-cafe0f67ced7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:11.635041 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e2da5-c368-49e9-ac4d-cafe0f67ced7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:11.637970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e2da5-c368-49e9-ac4d-cafe0f67ced7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:11.653082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1322/2636] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:11.657244 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e2da5-c368-49e9-ac4d-cafe0f67ced7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:11.684874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e2da5-c368-49e9-ac4d-cafe0f67ced7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:11.688880 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-28fc556e-6e82-44e7-98a8-d38a7e5848d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91b3a982f4484b6c9e8ef46494f4c92a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:11.695322 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c22e2d-ce13-49f8-b2aa-1e8a3a707aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:11.696063 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c22e2d-ce13-49f8-b2aa-1e8a3a707aae admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:11.701753 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c22e2d-ce13-49f8-b2aa-1e8a3a707aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:11.702545 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c22e2d-ce13-49f8-b2aa-1e8a3a707aae admin admin] PATH_INFO: `/v3/projects/5dce5d9a799a49c28ddfd7006b4709a1/users/74cc37c45b984d0c99e9746a82010c08/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:11.707174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-28fc556e-6e82-44e7-98a8-d38a7e5848d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91b3a982f4484b6c9e8ef46494f4c92a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:11.717940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-28fc556e-6e82-44e7-98a8-d38a7e5848d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91b3a982f4484b6c9e8ef46494f4c92a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:11.718779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-28fc556e-6e82-44e7-98a8-d38a7e5848d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91b3a982f4484b6c9e8ef46494f4c92a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:11.739527 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-28fc556e-6e82-44e7-98a8-d38a7e5848d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91b3a982f4484b6c9e8ef46494f4c92a'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:11.746050 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c22e2d-ce13-49f8-b2aa-1e8a3a707aae admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dce5d9a799a49c28ddfd7006b4709a1, user_id=74cc37c45b984d0c99e9746a82010c08, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:11.750480 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c22e2d-ce13-49f8-b2aa-1e8a3a707aae admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:11.756453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1315/2637] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:11 2022] DELETE /identity/v3/users/91b3a982f4484b6c9e8ef46494f4c92a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:11.783186 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1323/2638] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:11 2022] PUT /identity/v3/projects/5dce5d9a799a49c28ddfd7006b4709a1/users/74cc37c45b984d0c99e9746a82010c08/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:11.787865 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc556e-6e82-44e7-98a8-d38a7e5848d5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:11.789667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-28fc556e-6e82-44e7-98a8-d38a7e5848d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:11.868106 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc556e-6e82-44e7-98a8-d38a7e5848d5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:11.961987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f13045-3e8c-4ff2-8c01-ff3b6f0ca4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:11.962680 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f13045-3e8c-4ff2-8c01-ff3b6f0ca4dc admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:11.962966 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f13045-3e8c-4ff2-8c01-ff3b6f0ca4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:11.963241 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f13045-3e8c-4ff2-8c01-ff3b6f0ca4dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:11.964205 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f13045-3e8c-4ff2-8c01-ff3b6f0ca4dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:11.965736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f13045-3e8c-4ff2-8c01-ff3b6f0ca4dc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:11.981789 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1316/2639] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:11 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:11.983984 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c22e2d-ce13-49f8-b2aa-1e8a3a707aae admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:12.000028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c22e2d-ce13-49f8-b2aa-1e8a3a707aae admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:12.005854 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153f95ac-cb61-443f-8db7-b7b754817313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:12.006748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f95ac-cb61-443f-8db7-b7b754817313 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:12.006917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f95ac-cb61-443f-8db7-b7b754817313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:12.007104 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153f95ac-cb61-443f-8db7-b7b754817313 admin admin] PATH_INFO: `/v3/projects/5dce5d9a799a49c28ddfd7006b4709a1/users/74cc37c45b984d0c99e9746a82010c08/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:12.010599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f95ac-cb61-443f-8db7-b7b754817313 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dce5d9a799a49c28ddfd7006b4709a1, user_id=74cc37c45b984d0c99e9746a82010c08, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:12.012548 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f13045-3e8c-4ff2-8c01-ff3b6f0ca4dc admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:12.014330 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153f95ac-cb61-443f-8db7-b7b754817313 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:12.014722 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-25f13045-3e8c-4ff2-8c01-ff3b6f0ca4dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:12.029570 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1324/2640] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:11 2022] PUT /identity/v3/projects/5dce5d9a799a49c28ddfd7006b4709a1/users/74cc37c45b984d0c99e9746a82010c08/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:12.042208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad10a62-9766-4963-95f1-a963b4bed09b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:12.043420 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad10a62-9766-4963-95f1-a963b4bed09b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:12.044013 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad10a62-9766-4963-95f1-a963b4bed09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:12.046069 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f13045-3e8c-4ff2-8c01-ff3b6f0ca4dc admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:12.056144 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:12.085298 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8ad10a62-9766-4963-95f1-a963b4bed09b None None] MFA Rules not processed for user `74cc37c45b984d0c99e9746a82010c08`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:12.086585 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8ad10a62-9766-4963-95f1-a963b4bed09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:12.088846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:12.128604 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0939c2-031f-49a7-822a-21f5dd4ffe52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:12.129393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0939c2-031f-49a7-822a-21f5dd4ffe52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:12.129821 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0939c2-031f-49a7-822a-21f5dd4ffe52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:12.130173 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0939c2-031f-49a7-822a-21f5dd4ffe52 admin admin] PATH_INFO: `/v3/projects/04fc9eb053f64757a168c04252c7f44f` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:12.131494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0939c2-031f-49a7-822a-21f5dd4ffe52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04fc9eb053f64757a168c04252c7f44f)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:12.133614 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0939c2-031f-49a7-822a-21f5dd4ffe52 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:12.176534 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1325/2641] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:12 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:12.189372 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad10a62-9766-4963-95f1-a963b4bed09b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:12.190965 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8ad10a62-9766-4963-95f1-a963b4bed09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:12.204610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1f0939c2-031f-49a7-822a-21f5dd4ffe52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04fc9eb053f64757a168c04252c7f44f', 'request_id': 'req-1f0939c2-031f-49a7-822a-21f5dd4ffe52'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:12.217118 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1f0939c2-031f-49a7-822a-21f5dd4ffe52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04fc9eb053f64757a168c04252c7f44f', 'request_id': 'req-1f0939c2-031f-49a7-822a-21f5dd4ffe52'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:12.225168 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1f0939c2-031f-49a7-822a-21f5dd4ffe52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04fc9eb053f64757a168c04252c7f44f', 'request_id': 'req-1f0939c2-031f-49a7-822a-21f5dd4ffe52'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:12.243145 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1f0939c2-031f-49a7-822a-21f5dd4ffe52 admin admin] The token cache is being invalidate because project 04fc9eb053f64757a168c04252c7f44f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:12.243424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1f0939c2-031f-49a7-822a-21f5dd4ffe52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:12.244100 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-1f0939c2-031f-49a7-822a-21f5dd4ffe52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:12.249105 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1317/2642] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:12 2022] DELETE /identity/v3/projects/04fc9eb053f64757a168c04252c7f44f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:12.250737 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77d252-9977-490d-8bae-6f488b5dae40 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:12.251027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77d252-9977-490d-8bae-6f488b5dae40 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:12.251303 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77d252-9977-490d-8bae-6f488b5dae40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:12.272292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-ab77d252-9977-490d-8bae-6f488b5dae40 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:12.273443 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-ab77d252-9977-490d-8bae-6f488b5dae40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:12.273657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad10a62-9766-4963-95f1-a963b4bed09b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:12.338617 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30d2e0-edad-4dd3-b675-9c1207b7db86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:12.339478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30d2e0-edad-4dd3-b675-9c1207b7db86 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:12.339822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30d2e0-edad-4dd3-b675-9c1207b7db86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:12.340486 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b30d2e0-edad-4dd3-b675-9c1207b7db86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:12.342501 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5b30d2e0-edad-4dd3-b675-9c1207b7db86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:12.351687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1318/2643] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:36:12.359290 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfbb7b-b9e9-4d39-a794-dec69cdd2aae None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:12.359864 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfbb7b-b9e9-4d39-a794-dec69cdd2aae None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:12.360394 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfbb7b-b9e9-4d39-a794-dec69cdd2aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:12.388679 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-82bfbb7b-b9e9-4d39-a794-dec69cdd2aae None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:12.390330 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-82bfbb7b-b9e9-4d39-a794-dec69cdd2aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:12.407094 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1319/2644] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:36:12.408801 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bfbb7b-b9e9-4d39-a794-dec69cdd2aae None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:12.420686 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bfbb7b-b9e9-4d39-a794-dec69cdd2aae None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:12.427363 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a424dc-a5be-4c53-a1ed-0daed39b7fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:12.428004 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a424dc-a5be-4c53-a1ed-0daed39b7fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:12.428284 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a424dc-a5be-4c53-a1ed-0daed39b7fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:12.429055 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a424dc-a5be-4c53-a1ed-0daed39b7fd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:12.429301 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a424dc-a5be-4c53-a1ed-0daed39b7fd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:36:12.429718 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a424dc-a5be-4c53-a1ed-0daed39b7fd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:12.431185 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a424dc-a5be-4c53-a1ed-0daed39b7fd0 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:12.440999 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:12.445043 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30d2e0-edad-4dd3-b675-9c1207b7db86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:12.447848 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1320/2645] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:36:12.448426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b30d2e0-edad-4dd3-b675-9c1207b7db86 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:12.453464 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a424dc-a5be-4c53-a1ed-0daed39b7fd0 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:12.464505 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1326/2646] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:12 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:12.470620 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a424dc-a5be-4c53-a1ed-0daed39b7fd0 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:12.476413 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc2786-a495-4622-80d3-68b3ed7c4bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:12.477042 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc2786-a495-4622-80d3-68b3ed7c4bee admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:12.477330 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc2786-a495-4622-80d3-68b3ed7c4bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:12.477665 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc2786-a495-4622-80d3-68b3ed7c4bee admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:12.478337 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc2786-a495-4622-80d3-68b3ed7c4bee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:12.480256 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc2786-a495-4622-80d3-68b3ed7c4bee admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:12.525945 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1321/2647] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:12 2022] POST /identity/v3/projects => generated 371 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:12.530222 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30d2e0-edad-4dd3-b675-9c1207b7db86 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:12.541074 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc2786-a495-4622-80d3-68b3ed7c4bee admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:12.546490 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-a2cc2786-a495-4622-80d3-68b3ed7c4bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:12.547179 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b30d2e0-edad-4dd3-b675-9c1207b7db86 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:12.572884 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:12.637391 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc2786-a495-4622-80d3-68b3ed7c4bee admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:12.642158 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d6156-b937-49d9-9690-6af8dd075650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:12.642752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f585d2-e63f-420d-b872-7b34400fb6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:12.645126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d6156-b937-49d9-9690-6af8dd075650 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:12.645126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d6156-b937-49d9-9690-6af8dd075650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:12.645126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d6156-b937-49d9-9690-6af8dd075650 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:12.645455 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f585d2-e63f-420d-b872-7b34400fb6ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:12.645455 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f585d2-e63f-420d-b872-7b34400fb6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:12.645455 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f585d2-e63f-420d-b872-7b34400fb6ba admin admin] PATH_INFO: `/v3/users/bc5fb76567e64b13a4482ca6b9dbf996` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:12.645783 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d6156-b937-49d9-9690-6af8dd075650 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:12.646640 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f585d2-e63f-420d-b872-7b34400fb6ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc5fb76567e64b13a4482ca6b9dbf996)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:12.648218 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d6156-b937-49d9-9690-6af8dd075650 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:12.648991 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f585d2-e63f-420d-b872-7b34400fb6ba admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:12.686887 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1327/2648] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:12 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:12.693131 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d6156-b937-49d9-9690-6af8dd075650 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:12.711872 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d6156-b937-49d9-9690-6af8dd075650 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:12.713501 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-03f585d2-e63f-420d-b872-7b34400fb6ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc5fb76567e64b13a4482ca6b9dbf996'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:12.722130 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93d187-4bdc-4d5a-a600-35bd5d30eebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:12.723142 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93d187-4bdc-4d5a-a600-35bd5d30eebc admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:12.723700 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93d187-4bdc-4d5a-a600-35bd5d30eebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:12.727882 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-03f585d2-e63f-420d-b872-7b34400fb6ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc5fb76567e64b13a4482ca6b9dbf996'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:12.735856 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93d187-4bdc-4d5a-a600-35bd5d30eebc admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:12.738376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd93d187-4bdc-4d5a-a600-35bd5d30eebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:12.741680 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-03f585d2-e63f-420d-b872-7b34400fb6ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc5fb76567e64b13a4482ca6b9dbf996'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:12.741842 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-03f585d2-e63f-420d-b872-7b34400fb6ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc5fb76567e64b13a4482ca6b9dbf996'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:12.746481 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd93d187-4bdc-4d5a-a600-35bd5d30eebc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:12.759442 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-03f585d2-e63f-420d-b872-7b34400fb6ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc5fb76567e64b13a4482ca6b9dbf996'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:12.767334 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1322/2649] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:12 2022] DELETE /identity/v3/users/bc5fb76567e64b13a4482ca6b9dbf996 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:12.770753 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1328/2650] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:36:12.775528 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfec264-46de-47c7-bf10-9fe2b1ae54a4 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:12.775633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93d187-4bdc-4d5a-a600-35bd5d30eebc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:12.775727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfec264-46de-47c7-bf10-9fe2b1ae54a4 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:12.775727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfec264-46de-47c7-bf10-9fe2b1ae54a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:12.777453 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bd93d187-4bdc-4d5a-a600-35bd5d30eebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:12.813192 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93d187-4bdc-4d5a-a600-35bd5d30eebc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:12.814470 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-6dfec264-46de-47c7-bf10-9fe2b1ae54a4 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:12.816411 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6dfec264-46de-47c7-bf10-9fe2b1ae54a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:12.940229 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd37cf-4620-4eb3-aa7d-2c6b3fd8932b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:12.940950 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd37cf-4620-4eb3-aa7d-2c6b3fd8932b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:12.941286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd37cf-4620-4eb3-aa7d-2c6b3fd8932b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:12.941654 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cd37cf-4620-4eb3-aa7d-2c6b3fd8932b admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:12.942452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd37cf-4620-4eb3-aa7d-2c6b3fd8932b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:12.944209 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cd37cf-4620-4eb3-aa7d-2c6b3fd8932b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:12.960182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1329/2651] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:12.963016 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1323/2652] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:12.971721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd37cf-4620-4eb3-aa7d-2c6b3fd8932b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:12.981161 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfec264-46de-47c7-bf10-9fe2b1ae54a4 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:12.983114 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6dfec264-46de-47c7-bf10-9fe2b1ae54a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:12.994067 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cd37cf-4620-4eb3-aa7d-2c6b3fd8932b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:13.007670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efe1fa-ddf5-4a27-bd65-01fe5150a2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:13.012751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efe1fa-ddf5-4a27-bd65-01fe5150a2af admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:13.012751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efe1fa-ddf5-4a27-bd65-01fe5150a2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:13.012751 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efe1fa-ddf5-4a27-bd65-01fe5150a2af admin admin] PATH_INFO: `/v3/projects/b4beb065ba474c8594da3c4d1072026e/users/be4c24d2acc444de93cc573dbc096f1d/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:13.048980 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efe1fa-ddf5-4a27-bd65-01fe5150a2af admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4beb065ba474c8594da3c4d1072026e, user_id=be4c24d2acc444de93cc573dbc096f1d, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:13.054019 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66efe1fa-ddf5-4a27-bd65-01fe5150a2af admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:13.079388 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1330/2653] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:12 2022] PUT /identity/v3/projects/b4beb065ba474c8594da3c4d1072026e/users/be4c24d2acc444de93cc573dbc096f1d/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:13.092228 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5994a2-2f77-4c52-8d35-4636078d0566 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:13.092729 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5994a2-2f77-4c52-8d35-4636078d0566 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:13.093172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5994a2-2f77-4c52-8d35-4636078d0566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:13.103742 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:13.125136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfec264-46de-47c7-bf10-9fe2b1ae54a4 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:13.155855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0e5994a2-2f77-4c52-8d35-4636078d0566 None None] MFA Rules not processed for user `be4c24d2acc444de93cc573dbc096f1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:13.157559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0e5994a2-2f77-4c52-8d35-4636078d0566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:13.212459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053396f-f948-48c3-a456-dc80752b6a3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:13.213395 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053396f-f948-48c3-a456-dc80752b6a3c service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:13.213711 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053396f-f948-48c3-a456-dc80752b6a3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:13.213987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053396f-f948-48c3-a456-dc80752b6a3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:13.218216 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:13.308739 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053396f-f948-48c3-a456-dc80752b6a3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:13.310598 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053396f-f948-48c3-a456-dc80752b6a3c service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:13.318739 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1331/2654] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:36:13.326584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5994a2-2f77-4c52-8d35-4636078d0566 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:13.329095 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0e5994a2-2f77-4c52-8d35-4636078d0566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:13.330203 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1324/2655] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:13.434664 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5994a2-2f77-4c52-8d35-4636078d0566 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:13.441497 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9a02e-6315-4832-8ccf-ebc64ecf694d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fecb440c0874ede9f3cf544cfed93ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:13.442585 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9a02e-6315-4832-8ccf-ebc64ecf694d service glance] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:13.442868 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9a02e-6315-4832-8ccf-ebc64ecf694d service glance] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:13.443600 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf9a02e-6315-4832-8ccf-ebc64ecf694d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:13.457269 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9a02e-6315-4832-8ccf-ebc64ecf694d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:13.460027 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf9a02e-6315-4832-8ccf-ebc64ecf694d service glance] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:13.475473 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1332/2656] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:13.770712 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053396f-f948-48c3-a456-dc80752b6a3c service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:13.772407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e053396f-f948-48c3-a456-dc80752b6a3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:13.788947 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053396f-f948-48c3-a456-dc80752b6a3c service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:13.794934 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4e0a6-a85a-4db1-a7cd-c225297c1609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:13.795842 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4e0a6-a85a-4db1-a7cd-c225297c1609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:13.796290 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4e0a6-a85a-4db1-a7cd-c225297c1609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:13.796726 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4e0a6-a85a-4db1-a7cd-c225297c1609 admin admin] PATH_INFO: `/v3/projects/a5fd791817be4343bc045308735998bb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:13.799904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4e0a6-a85a-4db1-a7cd-c225297c1609 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5fd791817be4343bc045308735998bb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:13.807738 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf4e0a6-a85a-4db1-a7cd-c225297c1609 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:13.905903 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cdf4e0a6-a85a-4db1-a7cd-c225297c1609 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5fd791817be4343bc045308735998bb', 'request_id': 'req-cdf4e0a6-a85a-4db1-a7cd-c225297c1609'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:13.921430 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cdf4e0a6-a85a-4db1-a7cd-c225297c1609 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5fd791817be4343bc045308735998bb', 'request_id': 'req-cdf4e0a6-a85a-4db1-a7cd-c225297c1609'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:13.931571 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cdf4e0a6-a85a-4db1-a7cd-c225297c1609 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5fd791817be4343bc045308735998bb', 'request_id': 'req-cdf4e0a6-a85a-4db1-a7cd-c225297c1609'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:13.949877 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cdf4e0a6-a85a-4db1-a7cd-c225297c1609 admin admin] The token cache is being invalidate because project a5fd791817be4343bc045308735998bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:13.950349 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cdf4e0a6-a85a-4db1-a7cd-c225297c1609 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:13.951374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-cdf4e0a6-a85a-4db1-a7cd-c225297c1609 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:13.964129 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1325/2657] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:13 2022] DELETE /identity/v3/projects/a5fd791817be4343bc045308735998bb => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:13.965370 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9a02e-6315-4832-8ccf-ebc64ecf694d service glance] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:13.982921 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf9a02e-6315-4832-8ccf-ebc64ecf694d service glance] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:14.072051 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbcf926-ddfc-40ba-a8ed-23b8d3eab3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:14.072719 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcf926-ddfc-40ba-a8ed-23b8d3eab3b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:14.073048 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcf926-ddfc-40ba-a8ed-23b8d3eab3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:14.073337 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbcf926-ddfc-40ba-a8ed-23b8d3eab3b5 admin admin] PATH_INFO: `/v3/users/74cc37c45b984d0c99e9746a82010c08` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:14.075437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbcf926-ddfc-40ba-a8ed-23b8d3eab3b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74cc37c45b984d0c99e9746a82010c08)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:14.077653 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbcf926-ddfc-40ba-a8ed-23b8d3eab3b5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:14.153993 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6dbcf926-ddfc-40ba-a8ed-23b8d3eab3b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74cc37c45b984d0c99e9746a82010c08'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:14.164505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4e0a6-a85a-4db1-a7cd-c225297c1609 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:14.166030 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-cdf4e0a6-a85a-4db1-a7cd-c225297c1609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:14.174925 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6dbcf926-ddfc-40ba-a8ed-23b8d3eab3b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74cc37c45b984d0c99e9746a82010c08'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:14.175666 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6dbcf926-ddfc-40ba-a8ed-23b8d3eab3b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74cc37c45b984d0c99e9746a82010c08'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:14.177553 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6dbcf926-ddfc-40ba-a8ed-23b8d3eab3b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74cc37c45b984d0c99e9746a82010c08'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:14.193213 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4e0a6-a85a-4db1-a7cd-c225297c1609 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:14.199661 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58748e-430b-43e9-ac12-885290714450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:14.200331 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58748e-430b-43e9-ac12-885290714450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:14.200673 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58748e-430b-43e9-ac12-885290714450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:14.200956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58748e-430b-43e9-ac12-885290714450 admin admin] PATH_INFO: `/v3/users/1b0b33e7d4af482183de5099ca60e6a4` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:14.201120 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6dbcf926-ddfc-40ba-a8ed-23b8d3eab3b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74cc37c45b984d0c99e9746a82010c08'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:14.202283 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58748e-430b-43e9-ac12-885290714450 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b0b33e7d4af482183de5099ca60e6a4)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:14.204276 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58748e-430b-43e9-ac12-885290714450 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:14.215141 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:14.233272 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1333/2658] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:13 2022] DELETE /identity/v3/users/74cc37c45b984d0c99e9746a82010c08 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:14.254852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:14.289783 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9b58748e-430b-43e9-ac12-885290714450 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b0b33e7d4af482183de5099ca60e6a4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:14.300245 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9b58748e-430b-43e9-ac12-885290714450 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b0b33e7d4af482183de5099ca60e6a4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:14.309268 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9b58748e-430b-43e9-ac12-885290714450 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b0b33e7d4af482183de5099ca60e6a4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:14.310081 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9b58748e-430b-43e9-ac12-885290714450 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b0b33e7d4af482183de5099ca60e6a4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:14.325796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9b58748e-430b-43e9-ac12-885290714450 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b0b33e7d4af482183de5099ca60e6a4'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:14.333845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1326/2659] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:14 2022] DELETE /identity/v3/users/1b0b33e7d4af482183de5099ca60e6a4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:14.338401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423e5a2-ece6-4ffe-9d1d-f8a20ce4fe02 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:14.338809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423e5a2-ece6-4ffe-9d1d-f8a20ce4fe02 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:14.339205 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423e5a2-ece6-4ffe-9d1d-f8a20ce4fe02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:14.365472 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4423e5a2-ece6-4ffe-9d1d-f8a20ce4fe02 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:14.367320 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4423e5a2-ece6-4ffe-9d1d-f8a20ce4fe02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:14.447785 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1334/2660] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:14.459961 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58748e-430b-43e9-ac12-885290714450 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:14.462122 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-9b58748e-430b-43e9-ac12-885290714450 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:14.555660 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58748e-430b-43e9-ac12-885290714450 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:14.560936 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ffb91d-31b8-4832-a264-1a4b9cc54ff3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:14.561764 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffb91d-31b8-4832-a264-1a4b9cc54ff3 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:14.562093 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffb91d-31b8-4832-a264-1a4b9cc54ff3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:14.562435 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ffb91d-31b8-4832-a264-1a4b9cc54ff3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:14.573215 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4423e5a2-ece6-4ffe-9d1d-f8a20ce4fe02 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:14.574784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4423e5a2-ece6-4ffe-9d1d-f8a20ce4fe02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:14.587725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ffb91d-31b8-4832-a264-1a4b9cc54ff3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:14.591280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ffb91d-31b8-4832-a264-1a4b9cc54ff3 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:14.593806 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4423e5a2-ece6-4ffe-9d1d-f8a20ce4fe02 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:14.600308 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3728970-73b6-44af-8513-0ccc88e077f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:14.601051 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3728970-73b6-44af-8513-0ccc88e077f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:14.601422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3728970-73b6-44af-8513-0ccc88e077f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:14.601843 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3728970-73b6-44af-8513-0ccc88e077f7 admin admin] PATH_INFO: `/v3/projects/5dce5d9a799a49c28ddfd7006b4709a1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:14.603142 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3728970-73b6-44af-8513-0ccc88e077f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dce5d9a799a49c28ddfd7006b4709a1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:14.603710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1327/2661] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:14.605398 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3728970-73b6-44af-8513-0ccc88e077f7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:14.663187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b3728970-73b6-44af-8513-0ccc88e077f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dce5d9a799a49c28ddfd7006b4709a1', 'request_id': 'req-b3728970-73b6-44af-8513-0ccc88e077f7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:14.670711 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b3728970-73b6-44af-8513-0ccc88e077f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dce5d9a799a49c28ddfd7006b4709a1', 'request_id': 'req-b3728970-73b6-44af-8513-0ccc88e077f7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:14.677874 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b3728970-73b6-44af-8513-0ccc88e077f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dce5d9a799a49c28ddfd7006b4709a1', 'request_id': 'req-b3728970-73b6-44af-8513-0ccc88e077f7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:14.704871 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b3728970-73b6-44af-8513-0ccc88e077f7 admin admin] The token cache is being invalidate because project 5dce5d9a799a49c28ddfd7006b4709a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:14.705168 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b3728970-73b6-44af-8513-0ccc88e077f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:14.706238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b3728970-73b6-44af-8513-0ccc88e077f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:14.708125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1335/2662] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:14 2022] DELETE /identity/v3/projects/5dce5d9a799a49c28ddfd7006b4709a1 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:14.712705 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcbdb2-cecf-4a7a-b5d2-9a1e04d902ee None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:14.713024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcbdb2-cecf-4a7a-b5d2-9a1e04d902ee None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:14.713275 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edcbdb2-cecf-4a7a-b5d2-9a1e04d902ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:14.733505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5edcbdb2-cecf-4a7a-b5d2-9a1e04d902ee None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:14.734754 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5edcbdb2-cecf-4a7a-b5d2-9a1e04d902ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:14.817568 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1328/2663] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:36:14.820947 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548f1a4-3830-400a-b91b-234640f37900 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:14.821474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548f1a4-3830-400a-b91b-234640f37900 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:14.821992 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548f1a4-3830-400a-b91b-234640f37900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:14.852380 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6548f1a4-3830-400a-b91b-234640f37900 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:14.853748 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6548f1a4-3830-400a-b91b-234640f37900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:14.862247 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1336/2664] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:14.866077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edcbdb2-cecf-4a7a-b5d2-9a1e04d902ee None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:14.876070 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edcbdb2-cecf-4a7a-b5d2-9a1e04d902ee None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:14.881415 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04b7e8-bdbf-4b41-b081-f5a3db45a2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:14.882060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04b7e8-bdbf-4b41-b081-f5a3db45a2d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:14.882365 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04b7e8-bdbf-4b41-b081-f5a3db45a2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:14.882937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a04b7e8-bdbf-4b41-b081-f5a3db45a2d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:14.883356 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04b7e8-bdbf-4b41-b081-f5a3db45a2d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:36:14.883659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04b7e8-bdbf-4b41-b081-f5a3db45a2d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:14.885251 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a04b7e8-bdbf-4b41-b081-f5a3db45a2d6 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:14.897439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1329/2665] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:36:14.902670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548f1a4-3830-400a-b91b-234640f37900 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:14.911295 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04b7e8-bdbf-4b41-b081-f5a3db45a2d6 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:14.913336 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3a04b7e8-bdbf-4b41-b081-f5a3db45a2d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:14.915217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548f1a4-3830-400a-b91b-234640f37900 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:14.921236 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04611308-0722-437a-a03e-ed41da86587f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:14.921912 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04611308-0722-437a-a03e-ed41da86587f admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:14.922334 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04611308-0722-437a-a03e-ed41da86587f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:14.922654 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04611308-0722-437a-a03e-ed41da86587f admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:14.923344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04611308-0722-437a-a03e-ed41da86587f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:14.925456 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04611308-0722-437a-a03e-ed41da86587f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:14.930766 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a04b7e8-bdbf-4b41-b081-f5a3db45a2d6 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:14.936248 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf448bd-0d65-4fb8-b611-89b90c243dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:14.937065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf448bd-0d65-4fb8-b611-89b90c243dea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:14.937363 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf448bd-0d65-4fb8-b611-89b90c243dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:14.937696 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf448bd-0d65-4fb8-b611-89b90c243dea admin admin] PATH_INFO: `/v3/projects/983e5f7586e3435bb6442b41b467ddd1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:14.939015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf448bd-0d65-4fb8-b611-89b90c243dea admin admin] RBAC: Authorizing `identity:delete_project(project_id=983e5f7586e3435bb6442b41b467ddd1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:14.942346 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf448bd-0d65-4fb8-b611-89b90c243dea admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:14.976338 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1337/2666] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:14 2022] POST /identity/v3/projects => generated 387 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:14.980824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04611308-0722-437a-a03e-ed41da86587f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:14.990102 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:15.010806 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fcf448bd-0d65-4fb8-b611-89b90c243dea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '983e5f7586e3435bb6442b41b467ddd1', 'request_id': 'req-fcf448bd-0d65-4fb8-b611-89b90c243dea'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:15.038457 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fcf448bd-0d65-4fb8-b611-89b90c243dea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '983e5f7586e3435bb6442b41b467ddd1', 'request_id': 'req-fcf448bd-0d65-4fb8-b611-89b90c243dea'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:15.045744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:15.045901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fcf448bd-0d65-4fb8-b611-89b90c243dea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '983e5f7586e3435bb6442b41b467ddd1', 'request_id': 'req-fcf448bd-0d65-4fb8-b611-89b90c243dea'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:15.059426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04611308-0722-437a-a03e-ed41da86587f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:15.059925 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fcf448bd-0d65-4fb8-b611-89b90c243dea admin admin] The token cache is being invalidate because project 983e5f7586e3435bb6442b41b467ddd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:15.060215 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fcf448bd-0d65-4fb8-b611-89b90c243dea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:15.061047 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-fcf448bd-0d65-4fb8-b611-89b90c243dea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:15.067602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1330/2667] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:14 2022] DELETE /identity/v3/projects/983e5f7586e3435bb6442b41b467ddd1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:15.071303 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf448bd-0d65-4fb8-b611-89b90c243dea admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:15.072708 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-fcf448bd-0d65-4fb8-b611-89b90c243dea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:15.121399 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb36b7-e8d1-41b2-b302-62ba1e9c0109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:15.122333 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb36b7-e8d1-41b2-b302-62ba1e9c0109 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:15.122629 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb36b7-e8d1-41b2-b302-62ba1e9c0109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:15.122899 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb36b7-e8d1-41b2-b302-62ba1e9c0109 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:15.123535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb36b7-e8d1-41b2-b302-62ba1e9c0109 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:15.125438 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb36b7-e8d1-41b2-b302-62ba1e9c0109 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:15.140016 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf448bd-0d65-4fb8-b611-89b90c243dea admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:15.148620 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1338/2668] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:14 2022] POST /identity/v3/users => generated 294 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:15.151768 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc0ae5-b9bd-4c1a-8d29-989cc56ea0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:15.152763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb36b7-e8d1-41b2-b302-62ba1e9c0109 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:15.153105 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc0ae5-b9bd-4c1a-8d29-989cc56ea0d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:15.153397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc0ae5-b9bd-4c1a-8d29-989cc56ea0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:15.153869 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc0ae5-b9bd-4c1a-8d29-989cc56ea0d1 admin admin] PATH_INFO: `/v3/users/8b77687caf504842a748f0a4fd9ade54` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:15.154817 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-00bb36b7-e8d1-41b2-b302-62ba1e9c0109 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:15.162446 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc0ae5-b9bd-4c1a-8d29-989cc56ea0d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b77687caf504842a748f0a4fd9ade54)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:15.164551 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc0ae5-b9bd-4c1a-8d29-989cc56ea0d1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:15.169754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb36b7-e8d1-41b2-b302-62ba1e9c0109 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:15.174901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134cdb3-fcda-4728-93a6-24a93b7c9725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:15.175290 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134cdb3-fcda-4728-93a6-24a93b7c9725 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:15.175572 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134cdb3-fcda-4728-93a6-24a93b7c9725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:15.175842 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134cdb3-fcda-4728-93a6-24a93b7c9725 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:15.176567 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1134cdb3-fcda-4728-93a6-24a93b7c9725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:15.178203 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1134cdb3-fcda-4728-93a6-24a93b7c9725 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:15.188993 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1339/2669] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:15.193716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134cdb3-fcda-4728-93a6-24a93b7c9725 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:15.204183 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134cdb3-fcda-4728-93a6-24a93b7c9725 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:15.209412 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d365c-3869-4390-b23f-a370a3a4eedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:15.210140 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d365c-3869-4390-b23f-a370a3a4eedc admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:15.210418 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d365c-3869-4390-b23f-a370a3a4eedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:15.210690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580d365c-3869-4390-b23f-a370a3a4eedc admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:15.211374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d365c-3869-4390-b23f-a370a3a4eedc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:15.212950 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580d365c-3869-4390-b23f-a370a3a4eedc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:15.213760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3cdc0ae5-b9bd-4c1a-8d29-989cc56ea0d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b77687caf504842a748f0a4fd9ade54'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:15.223444 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3cdc0ae5-b9bd-4c1a-8d29-989cc56ea0d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b77687caf504842a748f0a4fd9ade54'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:15.225295 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1340/2670] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:15.230237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d365c-3869-4390-b23f-a370a3a4eedc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:15.230844 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3cdc0ae5-b9bd-4c1a-8d29-989cc56ea0d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b77687caf504842a748f0a4fd9ade54'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:15.231512 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3cdc0ae5-b9bd-4c1a-8d29-989cc56ea0d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b77687caf504842a748f0a4fd9ade54'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:15.249749 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580d365c-3869-4390-b23f-a370a3a4eedc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:15.254959 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171aadd8-ba43-4bcc-810c-b72564f08c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:15.255808 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aadd8-ba43-4bcc-810c-b72564f08c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:15.256176 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aadd8-ba43-4bcc-810c-b72564f08c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:15.256508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171aadd8-ba43-4bcc-810c-b72564f08c99 admin admin] PATH_INFO: `/v3/projects/a9fa7d2c1f9d457fb868dc526d73d9ce/users/b9c013a9760047d3b513f1dc81dc3558/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:15.269422 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3cdc0ae5-b9bd-4c1a-8d29-989cc56ea0d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b77687caf504842a748f0a4fd9ade54'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:15.277109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1331/2671] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:15 2022] DELETE /identity/v3/users/8b77687caf504842a748f0a4fd9ade54 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:15.282582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171aadd8-ba43-4bcc-810c-b72564f08c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9fa7d2c1f9d457fb868dc526d73d9ce, user_id=b9c013a9760047d3b513f1dc81dc3558, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:15.286534 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171aadd8-ba43-4bcc-810c-b72564f08c99 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:15.313851 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1341/2672] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:15 2022] PUT /identity/v3/projects/a9fa7d2c1f9d457fb868dc526d73d9ce/users/b9c013a9760047d3b513f1dc81dc3558/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:15.326411 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2319f3a-8eb1-4df0-9638-17a3e0a49f83 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:15.327168 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2319f3a-8eb1-4df0-9638-17a3e0a49f83 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:15.327451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2319f3a-8eb1-4df0-9638-17a3e0a49f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:15.357200 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f2319f3a-8eb1-4df0-9638-17a3e0a49f83 None None] MFA Rules not processed for user `b9c013a9760047d3b513f1dc81dc3558`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:15.358337 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f2319f3a-8eb1-4df0-9638-17a3e0a49f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:15.381514 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:15.450811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1332/2673] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:15.455408 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171aadd8-ba43-4bcc-810c-b72564f08c99 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:15.458194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-171aadd8-ba43-4bcc-810c-b72564f08c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:15.524932 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171aadd8-ba43-4bcc-810c-b72564f08c99 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:15.530383 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5816291-93f9-4bbe-a9ed-737ca1e8ee72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:15.531293 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5816291-93f9-4bbe-a9ed-737ca1e8ee72 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:15.531766 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5816291-93f9-4bbe-a9ed-737ca1e8ee72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:15.532046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5816291-93f9-4bbe-a9ed-737ca1e8ee72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:15.533000 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5816291-93f9-4bbe-a9ed-737ca1e8ee72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:15.534984 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5816291-93f9-4bbe-a9ed-737ca1e8ee72 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:15.569249 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1342/2674] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:15 2022] POST /identity/v3/projects => generated 389 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:15.573937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2319f3a-8eb1-4df0-9638-17a3e0a49f83 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:15.589702 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5816291-93f9-4bbe-a9ed-737ca1e8ee72 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:15.591439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2319f3a-8eb1-4df0-9638-17a3e0a49f83 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:15.591901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d5816291-93f9-4bbe-a9ed-737ca1e8ee72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:15.635890 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:15.665447 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded91757-8a51-4ef2-822f-9a4761cf785c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:15.666137 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded91757-8a51-4ef2-822f-9a4761cf785c admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:15.666421 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded91757-8a51-4ef2-822f-9a4761cf785c admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:15.666728 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded91757-8a51-4ef2-822f-9a4761cf785c admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:15.667420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded91757-8a51-4ef2-822f-9a4761cf785c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:15.669444 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded91757-8a51-4ef2-822f-9a4761cf785c admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:15.680496 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5816291-93f9-4bbe-a9ed-737ca1e8ee72 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:15.686511 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d8cb3-dde9-48b1-9670-35add30cf570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:15.687155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d8cb3-dde9-48b1-9670-35add30cf570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:15.687450 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d8cb3-dde9-48b1-9670-35add30cf570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:15.687731 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d8cb3-dde9-48b1-9670-35add30cf570 admin admin] PATH_INFO: `/v3/projects/fd6d9a481f8b4c4b9dd9dad199f2160f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:15.688906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d8cb3-dde9-48b1-9670-35add30cf570 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd6d9a481f8b4c4b9dd9dad199f2160f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:15.691366 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d8cb3-dde9-48b1-9670-35add30cf570 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:15.693614 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1333/2675] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:15 2022] POST /identity/v3/users => generated 301 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:15.697645 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded91757-8a51-4ef2-822f-9a4761cf785c admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:15.709550 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded91757-8a51-4ef2-822f-9a4761cf785c admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:15.714135 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13b974-f5b2-4d7e-a654-453742c0f088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:15.714739 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13b974-f5b2-4d7e-a654-453742c0f088 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:15.715024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13b974-f5b2-4d7e-a654-453742c0f088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:15.715305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f13b974-f5b2-4d7e-a654-453742c0f088 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:15.715995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13b974-f5b2-4d7e-a654-453742c0f088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:15.717615 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f13b974-f5b2-4d7e-a654-453742c0f088 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:15.729001 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1334/2676] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:36:15.733103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13b974-f5b2-4d7e-a654-453742c0f088 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:15.741183 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bf7d8cb3-dde9-48b1-9670-35add30cf570 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd6d9a481f8b4c4b9dd9dad199f2160f', 'request_id': 'req-bf7d8cb3-dde9-48b1-9670-35add30cf570'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:15.744416 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f13b974-f5b2-4d7e-a654-453742c0f088 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:15.747464 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bf7d8cb3-dde9-48b1-9670-35add30cf570 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd6d9a481f8b4c4b9dd9dad199f2160f', 'request_id': 'req-bf7d8cb3-dde9-48b1-9670-35add30cf570'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:15.753698 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bf7d8cb3-dde9-48b1-9670-35add30cf570 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd6d9a481f8b4c4b9dd9dad199f2160f', 'request_id': 'req-bf7d8cb3-dde9-48b1-9670-35add30cf570'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:15.773061 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bf7d8cb3-dde9-48b1-9670-35add30cf570 admin admin] The token cache is being invalidate because project fd6d9a481f8b4c4b9dd9dad199f2160f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:15.773357 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bf7d8cb3-dde9-48b1-9670-35add30cf570 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:15.774072 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-bf7d8cb3-dde9-48b1-9670-35add30cf570 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:15.775864 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1343/2677] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:15 2022] DELETE /identity/v3/projects/fd6d9a481f8b4c4b9dd9dad199f2160f => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:15.781382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b974f-a5a6-4596-8b6d-b8608790ddb3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:15.781703 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b974f-a5a6-4596-8b6d-b8608790ddb3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:15.781985 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b974f-a5a6-4596-8b6d-b8608790ddb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:15.802827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ae9b974f-a5a6-4596-8b6d-b8608790ddb3 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:15.804035 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ae9b974f-a5a6-4596-8b6d-b8608790ddb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:15.814001 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a63c6-2e66-4dcf-8177-fca4f6ffeea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:15.814760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a63c6-2e66-4dcf-8177-fca4f6ffeea9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:15.816103 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a63c6-2e66-4dcf-8177-fca4f6ffeea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:15.816393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19a63c6-2e66-4dcf-8177-fca4f6ffeea9 admin admin] PATH_INFO: `/v3/projects/46d58da0a4884737bf1e08bfa8508d7b/users/318531fe577540a79f90b0a48f259ca4/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:15.832703 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a63c6-2e66-4dcf-8177-fca4f6ffeea9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46d58da0a4884737bf1e08bfa8508d7b, user_id=318531fe577540a79f90b0a48f259ca4, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:15.835439 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19a63c6-2e66-4dcf-8177-fca4f6ffeea9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:15.849288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1335/2678] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:15 2022] PUT /identity/v3/projects/46d58da0a4884737bf1e08bfa8508d7b/users/318531fe577540a79f90b0a48f259ca4/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:15.853116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a63c6-2e66-4dcf-8177-fca4f6ffeea9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:15.854784 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c19a63c6-2e66-4dcf-8177-fca4f6ffeea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:15.866906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1344/2679] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:36:15.875800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683c36f-8a25-44f7-83d3-ff7d04f94b61 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:15.876096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683c36f-8a25-44f7-83d3-ff7d04f94b61 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:15.876374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683c36f-8a25-44f7-83d3-ff7d04f94b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:15.885172 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:15.899034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3683c36f-8a25-44f7-83d3-ff7d04f94b61 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:15.900247 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3683c36f-8a25-44f7-83d3-ff7d04f94b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:15.926847 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19a63c6-2e66-4dcf-8177-fca4f6ffeea9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:15.931871 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a276d1-a7f3-4443-88c4-7bd4cc090201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:15.932513 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a276d1-a7f3-4443-88c4-7bd4cc090201 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:15.932801 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a276d1-a7f3-4443-88c4-7bd4cc090201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:15.933112 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a276d1-a7f3-4443-88c4-7bd4cc090201 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:15.933979 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a276d1-a7f3-4443-88c4-7bd4cc090201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:15.935746 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a276d1-a7f3-4443-88c4-7bd4cc090201 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:15.950995 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1336/2680] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:15 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:36:15.952015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a276d1-a7f3-4443-88c4-7bd4cc090201 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:15.963825 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a276d1-a7f3-4443-88c4-7bd4cc090201 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:15.965291 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:15.969979 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163cdaa-7a07-400a-9bcb-336ddfd212b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:15.970763 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163cdaa-7a07-400a-9bcb-336ddfd212b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:15.971218 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163cdaa-7a07-400a-9bcb-336ddfd212b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:15.971548 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163cdaa-7a07-400a-9bcb-336ddfd212b1 admin admin] PATH_INFO: `/v3/projects/46d58da0a4884737bf1e08bfa8508d7b/users/318531fe577540a79f90b0a48f259ca4/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:15.974085 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163cdaa-7a07-400a-9bcb-336ddfd212b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46d58da0a4884737bf1e08bfa8508d7b, user_id=318531fe577540a79f90b0a48f259ca4, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:15.976048 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1345/2681] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:36:15.977409 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163cdaa-7a07-400a-9bcb-336ddfd212b1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:15.980193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683c36f-8a25-44f7-83d3-ff7d04f94b61 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:15.988950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1337/2682] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:15 2022] PUT /identity/v3/projects/46d58da0a4884737bf1e08bfa8508d7b/users/318531fe577540a79f90b0a48f259ca4/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:15.990082 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683c36f-8a25-44f7-83d3-ff7d04f94b61 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:15.998921 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a1e55-0817-4db7-b1f0-3cd67670b13f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:15.999220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a1e55-0817-4db7-b1f0-3cd67670b13f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:15.999497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a1e55-0817-4db7-b1f0-3cd67670b13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:16.029288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-b33a1e55-0817-4db7-b1f0-3cd67670b13f None None] MFA Rules not processed for user `318531fe577540a79f90b0a48f259ca4`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:16.030725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b33a1e55-0817-4db7-b1f0-3cd67670b13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:16.053183 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea616ddf-0a52-4887-86f2-541034b90e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:16.053844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea616ddf-0a52-4887-86f2-541034b90e27 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:16.054125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea616ddf-0a52-4887-86f2-541034b90e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:16.054400 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea616ddf-0a52-4887-86f2-541034b90e27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:16.055099 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea616ddf-0a52-4887-86f2-541034b90e27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:36:16.055403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea616ddf-0a52-4887-86f2-541034b90e27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:16.057006 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea616ddf-0a52-4887-86f2-541034b90e27 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:16.071117 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1346/2683] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:36:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:36:16.075819 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea616ddf-0a52-4887-86f2-541034b90e27 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:16.086855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea616ddf-0a52-4887-86f2-541034b90e27 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:16.091664 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48de3ea-ce93-43d4-92bf-238bc32c3ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:16.092280 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48de3ea-ce93-43d4-92bf-238bc32c3ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:16.092564 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48de3ea-ce93-43d4-92bf-238bc32c3ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:16.092843 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48de3ea-ce93-43d4-92bf-238bc32c3ac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:16.093500 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48de3ea-ce93-43d4-92bf-238bc32c3ac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:16.095460 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48de3ea-ce93-43d4-92bf-238bc32c3ac4 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:16.118846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1338/2684] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:15 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:36:16.135983 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a1e55-0817-4db7-b1f0-3cd67670b13f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:16.136453 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1347/2685] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:16 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:36:16.138579 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b33a1e55-0817-4db7-b1f0-3cd67670b13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:16.140514 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48de3ea-ce93-43d4-92bf-238bc32c3ac4 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:16.200111 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a1e55-0817-4db7-b1f0-3cd67670b13f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:16.216755 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48de3ea-ce93-43d4-92bf-238bc32c3ac4 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:16.222898 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230d1c9-0c42-4b9f-a070-9e9f97f8066c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:16.223588 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230d1c9-0c42-4b9f-a070-9e9f97f8066c admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:16.223936 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230d1c9-0c42-4b9f-a070-9e9f97f8066c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:16.224328 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230d1c9-0c42-4b9f-a070-9e9f97f8066c admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:16.225071 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230d1c9-0c42-4b9f-a070-9e9f97f8066c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:16.227474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230d1c9-0c42-4b9f-a070-9e9f97f8066c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:16.247783 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:16.249425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1348/2686] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:16 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:36:16.254922 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230d1c9-0c42-4b9f-a070-9e9f97f8066c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:16.257643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce96a49-5aa4-4cdb-88e2-41856cad99f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:16.258453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce96a49-5aa4-4cdb-88e2-41856cad99f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:16.258911 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce96a49-5aa4-4cdb-88e2-41856cad99f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:16.259478 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce96a49-5aa4-4cdb-88e2-41856cad99f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:16.264883 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230d1c9-0c42-4b9f-a070-9e9f97f8066c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:16.270351 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d508d-c19e-4413-915a-4d41ec0989bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:16.270963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d508d-c19e-4413-915a-4d41ec0989bf admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:16.271235 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d508d-c19e-4413-915a-4d41ec0989bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:16.271509 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d508d-c19e-4413-915a-4d41ec0989bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:16.272198 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d508d-c19e-4413-915a-4d41ec0989bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:16.273810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d508d-c19e-4413-915a-4d41ec0989bf admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:16.287645 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1349/2687] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:36:16.288682 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d508d-c19e-4413-915a-4d41ec0989bf admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:16.298169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d508d-c19e-4413-915a-4d41ec0989bf admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:16.303331 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677df89-7ca5-478b-9d1c-aacf49455028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:16.303917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677df89-7ca5-478b-9d1c-aacf49455028 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:16.304200 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677df89-7ca5-478b-9d1c-aacf49455028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:16.304568 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a677df89-7ca5-478b-9d1c-aacf49455028 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:16.305254 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a677df89-7ca5-478b-9d1c-aacf49455028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:16.306864 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a677df89-7ca5-478b-9d1c-aacf49455028 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:16.332640 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1350/2688] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:16.333025 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce96a49-5aa4-4cdb-88e2-41856cad99f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:16.333880 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677df89-7ca5-478b-9d1c-aacf49455028 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:16.334646 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce96a49-5aa4-4cdb-88e2-41856cad99f2 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:16.346784 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1339/2689] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:16 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:36:16.346963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a677df89-7ca5-478b-9d1c-aacf49455028 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:16.351512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193e058-8b1b-4b83-905b-c4fd483831e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:16.352182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193e058-8b1b-4b83-905b-c4fd483831e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:16.352470 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193e058-8b1b-4b83-905b-c4fd483831e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:16.352753 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193e058-8b1b-4b83-905b-c4fd483831e6 admin admin] PATH_INFO: `/v3/projects/30408255d29a4c8cbf34be91128fd182/users/62fe654d142e4301a298c0a4b474e095/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:16.375827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193e058-8b1b-4b83-905b-c4fd483831e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30408255d29a4c8cbf34be91128fd182, user_id=62fe654d142e4301a298c0a4b474e095, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:16.378929 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a193e058-8b1b-4b83-905b-c4fd483831e6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:16.393948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1351/2690] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:16 2022] PUT /identity/v3/projects/30408255d29a4c8cbf34be91128fd182/users/62fe654d142e4301a298c0a4b474e095/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:16.404825 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa80d66-fb06-47dd-8457-292eb20928a5 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:16.405194 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa80d66-fb06-47dd-8457-292eb20928a5 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:16.405628 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa80d66-fb06-47dd-8457-292eb20928a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:16.441496 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-bfa80d66-fb06-47dd-8457-292eb20928a5 None None] MFA Rules not processed for user `62fe654d142e4301a298c0a4b474e095`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:16.442797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-bfa80d66-fb06-47dd-8457-292eb20928a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:16.504159 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193e058-8b1b-4b83-905b-c4fd483831e6 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:16.516533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1340/2691] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:16.524564 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:16.533048 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa80d66-fb06-47dd-8457-292eb20928a5 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:16.573166 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193e058-8b1b-4b83-905b-c4fd483831e6 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:16.577967 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31212fa7-2b02-431e-a3ec-fc43778b29be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:16.578606 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31212fa7-2b02-431e-a3ec-fc43778b29be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:16.578887 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31212fa7-2b02-431e-a3ec-fc43778b29be admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:16.579169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31212fa7-2b02-431e-a3ec-fc43778b29be admin admin] PATH_INFO: `/v3/users/b9c013a9760047d3b513f1dc81dc3558` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:16.581108 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31212fa7-2b02-431e-a3ec-fc43778b29be admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9c013a9760047d3b513f1dc81dc3558)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:16.583650 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31212fa7-2b02-431e-a3ec-fc43778b29be admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:16.597803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa80d66-fb06-47dd-8457-292eb20928a5 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:16.601721 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905e5c9d-b787-470e-b2b4-833360a66368 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:16.602511 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e5c9d-b787-470e-b2b4-833360a66368 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:16.602802 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e5c9d-b787-470e-b2b4-833360a66368 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:16.603082 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905e5c9d-b787-470e-b2b4-833360a66368 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:16.612653 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e5c9d-b787-470e-b2b4-833360a66368 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:16.614258 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905e5c9d-b787-470e-b2b4-833360a66368 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:16.625388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1341/2692] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:36:16.640934 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-31212fa7-2b02-431e-a3ec-fc43778b29be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9c013a9760047d3b513f1dc81dc3558'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:16.648660 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-31212fa7-2b02-431e-a3ec-fc43778b29be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9c013a9760047d3b513f1dc81dc3558'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:16.649410 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-31212fa7-2b02-431e-a3ec-fc43778b29be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9c013a9760047d3b513f1dc81dc3558'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:16.650122 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-31212fa7-2b02-431e-a3ec-fc43778b29be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9c013a9760047d3b513f1dc81dc3558'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:16.662714 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-31212fa7-2b02-431e-a3ec-fc43778b29be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9c013a9760047d3b513f1dc81dc3558'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:16.677219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1352/2693] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:16 2022] DELETE /identity/v3/users/b9c013a9760047d3b513f1dc81dc3558 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:16.682018 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e07093-4010-45de-9b0a-5a83961c3464 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:16.682332 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e07093-4010-45de-9b0a-5a83961c3464 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:16.683511 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e07093-4010-45de-9b0a-5a83961c3464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:16.702372 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-35e07093-4010-45de-9b0a-5a83961c3464 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:16.703686 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-35e07093-4010-45de-9b0a-5a83961c3464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:16.731426 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:16.777008 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1342/2694] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:16.787841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31212fa7-2b02-431e-a3ec-fc43778b29be admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:16.789810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-31212fa7-2b02-431e-a3ec-fc43778b29be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:16.850518 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31212fa7-2b02-431e-a3ec-fc43778b29be admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:16.854119 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89a20e-9b42-42e4-8ff8-ecc287bbe936 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:16.854731 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89a20e-9b42-42e4-8ff8-ecc287bbe936 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:16.855016 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89a20e-9b42-42e4-8ff8-ecc287bbe936 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:16.855290 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89a20e-9b42-42e4-8ff8-ecc287bbe936 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:16.864838 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b89a20e-9b42-42e4-8ff8-ecc287bbe936 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:16.866301 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b89a20e-9b42-42e4-8ff8-ecc287bbe936 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:16.876384 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1353/2695] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:17.178619 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e07093-4010-45de-9b0a-5a83961c3464 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:17.180993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-35e07093-4010-45de-9b0a-5a83961c3464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:17.204727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e07093-4010-45de-9b0a-5a83961c3464 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:17.210501 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f275e7-7975-4982-871f-6d9b8540b1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:17.211184 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f275e7-7975-4982-871f-6d9b8540b1b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:17.211465 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f275e7-7975-4982-871f-6d9b8540b1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:17.211770 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f275e7-7975-4982-871f-6d9b8540b1b8 admin admin] PATH_INFO: `/v3/projects/a9fa7d2c1f9d457fb868dc526d73d9ce` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:17.212954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f275e7-7975-4982-871f-6d9b8540b1b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9fa7d2c1f9d457fb868dc526d73d9ce)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:17.214907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f275e7-7975-4982-871f-6d9b8540b1b8 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:17.263755 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-05f275e7-7975-4982-871f-6d9b8540b1b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9fa7d2c1f9d457fb868dc526d73d9ce', 'request_id': 'req-05f275e7-7975-4982-871f-6d9b8540b1b8'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:17.270665 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-05f275e7-7975-4982-871f-6d9b8540b1b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9fa7d2c1f9d457fb868dc526d73d9ce', 'request_id': 'req-05f275e7-7975-4982-871f-6d9b8540b1b8'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:17.276126 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-05f275e7-7975-4982-871f-6d9b8540b1b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9fa7d2c1f9d457fb868dc526d73d9ce', 'request_id': 'req-05f275e7-7975-4982-871f-6d9b8540b1b8'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:17.287822 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-05f275e7-7975-4982-871f-6d9b8540b1b8 admin admin] The token cache is being invalidate because project a9fa7d2c1f9d457fb868dc526d73d9ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:17.288113 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-05f275e7-7975-4982-871f-6d9b8540b1b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:17.288767 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-05f275e7-7975-4982-871f-6d9b8540b1b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:17.290315 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1343/2696] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:17 2022] DELETE /identity/v3/projects/a9fa7d2c1f9d457fb868dc526d73d9ce => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:17.294256 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89a20e-9b42-42e4-8ff8-ecc287bbe936 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:17.303954 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89a20e-9b42-42e4-8ff8-ecc287bbe936 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:17.337396 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:17.362837 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8d9abe-35d3-4cbd-ae85-258314fa5103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:17.363474 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d9abe-35d3-4cbd-ae85-258314fa5103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:17.363761 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d9abe-35d3-4cbd-ae85-258314fa5103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:17.364038 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8d9abe-35d3-4cbd-ae85-258314fa5103 admin admin] PATH_INFO: `/v3/users/318531fe577540a79f90b0a48f259ca4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:17.365209 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d9abe-35d3-4cbd-ae85-258314fa5103 admin admin] RBAC: Authorizing `identity:delete_user(user_id=318531fe577540a79f90b0a48f259ca4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:17.367169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8d9abe-35d3-4cbd-ae85-258314fa5103 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:17.425672 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2d8d9abe-35d3-4cbd-ae85-258314fa5103 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '318531fe577540a79f90b0a48f259ca4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:17.433327 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2d8d9abe-35d3-4cbd-ae85-258314fa5103 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '318531fe577540a79f90b0a48f259ca4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:17.434069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2d8d9abe-35d3-4cbd-ae85-258314fa5103 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '318531fe577540a79f90b0a48f259ca4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:17.434664 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2d8d9abe-35d3-4cbd-ae85-258314fa5103 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '318531fe577540a79f90b0a48f259ca4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:17.446331 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2d8d9abe-35d3-4cbd-ae85-258314fa5103 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '318531fe577540a79f90b0a48f259ca4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:17.459752 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1354/2697] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:17 2022] DELETE /identity/v3/users/318531fe577540a79f90b0a48f259ca4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:17.735786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f275e7-7975-4982-871f-6d9b8540b1b8 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:17.737750 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-05f275e7-7975-4982-871f-6d9b8540b1b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:17.751341 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f275e7-7975-4982-871f-6d9b8540b1b8 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:17.813641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476b77a-f321-4171-a48c-4b7911266723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:17.814295 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476b77a-f321-4171-a48c-4b7911266723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:17.815128 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476b77a-f321-4171-a48c-4b7911266723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:17.815424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476b77a-f321-4171-a48c-4b7911266723 admin admin] PATH_INFO: `/v3/projects/46d58da0a4884737bf1e08bfa8508d7b` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:17.816640 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476b77a-f321-4171-a48c-4b7911266723 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46d58da0a4884737bf1e08bfa8508d7b)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:17.818599 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476b77a-f321-4171-a48c-4b7911266723 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:17.865368 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c476b77a-f321-4171-a48c-4b7911266723 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46d58da0a4884737bf1e08bfa8508d7b', 'request_id': 'req-c476b77a-f321-4171-a48c-4b7911266723'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:17.872344 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c476b77a-f321-4171-a48c-4b7911266723 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46d58da0a4884737bf1e08bfa8508d7b', 'request_id': 'req-c476b77a-f321-4171-a48c-4b7911266723'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:17.873699 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:17.879881 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c476b77a-f321-4171-a48c-4b7911266723 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46d58da0a4884737bf1e08bfa8508d7b', 'request_id': 'req-c476b77a-f321-4171-a48c-4b7911266723'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:17.891426 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c476b77a-f321-4171-a48c-4b7911266723 admin admin] The token cache is being invalidate because project 46d58da0a4884737bf1e08bfa8508d7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:17.891709 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c476b77a-f321-4171-a48c-4b7911266723 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:17.892328 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-c476b77a-f321-4171-a48c-4b7911266723 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:17.894490 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1344/2698] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:17 2022] DELETE /identity/v3/projects/46d58da0a4884737bf1e08bfa8508d7b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:17.899828 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bf481-31ea-40e2-8104-10e9be72c225 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:17.900207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bf481-31ea-40e2-8104-10e9be72c225 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:17.900490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bf481-31ea-40e2-8104-10e9be72c225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:17.921405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-883bf481-31ea-40e2-8104-10e9be72c225 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:17.922660 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-883bf481-31ea-40e2-8104-10e9be72c225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:17.992054 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1355/2699] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:18.058327 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991ece6-401c-44a1-85bf-712a3bc8646f None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:18.058615 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991ece6-401c-44a1-85bf-712a3bc8646f None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:18.058892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991ece6-401c-44a1-85bf-712a3bc8646f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:18.081993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-0991ece6-401c-44a1-85bf-712a3bc8646f None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:18.083234 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0991ece6-401c-44a1-85bf-712a3bc8646f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:18.091482 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1345/2700] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:18.096000 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bf481-31ea-40e2-8104-10e9be72c225 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:18.106216 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bf481-31ea-40e2-8104-10e9be72c225 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:18.111020 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4c5e3-287e-4f0b-9bb4-04cc0ab33a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:18.112365 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4c5e3-287e-4f0b-9bb4-04cc0ab33a08 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:18.112650 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4c5e3-287e-4f0b-9bb4-04cc0ab33a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:18.112938 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4c5e3-287e-4f0b-9bb4-04cc0ab33a08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:18.113651 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4c5e3-287e-4f0b-9bb4-04cc0ab33a08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:36:18.113956 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4c5e3-287e-4f0b-9bb4-04cc0ab33a08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:18.115723 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4c5e3-287e-4f0b-9bb4-04cc0ab33a08 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:18.128877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1356/2701] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:36:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:36:18.133281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991ece6-401c-44a1-85bf-712a3bc8646f None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:18.142917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991ece6-401c-44a1-85bf-712a3bc8646f None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:18.148009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96d5a2-189b-4d04-8dda-5641993e6450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:18.148710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96d5a2-189b-4d04-8dda-5641993e6450 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:18.148990 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96d5a2-189b-4d04-8dda-5641993e6450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:18.149260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96d5a2-189b-4d04-8dda-5641993e6450 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:18.149982 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96d5a2-189b-4d04-8dda-5641993e6450 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:18.151888 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab96d5a2-189b-4d04-8dda-5641993e6450 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:18.179748 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1346/2702] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:18 2022] POST /identity/v3/projects => generated 409 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:18.184176 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4c5e3-287e-4f0b-9bb4-04cc0ab33a08 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:18.195511 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4c5e3-287e-4f0b-9bb4-04cc0ab33a08 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:18.254044 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:18.273449 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c609b2f-b35b-4f43-bacd-357981856ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:18.274126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c609b2f-b35b-4f43-bacd-357981856ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:18.274419 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c609b2f-b35b-4f43-bacd-357981856ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:18.274697 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c609b2f-b35b-4f43-bacd-357981856ed5 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:18.275361 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c609b2f-b35b-4f43-bacd-357981856ed5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:18.277275 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c609b2f-b35b-4f43-bacd-357981856ed5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:18.298670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1357/2703] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:18 2022] POST /identity/v3/users => generated 305 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:18.303494 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96d5a2-189b-4d04-8dda-5641993e6450 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:18.313760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96d5a2-189b-4d04-8dda-5641993e6450 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:18.318930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519116b-f6a0-4ed1-bc8f-384a0081461b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:18.319534 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519116b-f6a0-4ed1-bc8f-384a0081461b admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:18.319815 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519116b-f6a0-4ed1-bc8f-384a0081461b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:18.320090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519116b-f6a0-4ed1-bc8f-384a0081461b admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:18.320805 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519116b-f6a0-4ed1-bc8f-384a0081461b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:18.322423 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519116b-f6a0-4ed1-bc8f-384a0081461b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:18.333200 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1347/2704] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:18.337098 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c609b2f-b35b-4f43-bacd-357981856ed5 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:18.347263 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c609b2f-b35b-4f43-bacd-357981856ed5 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:18.351926 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611ddf39-6ebe-42b3-9d10-6c679764fb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:18.352526 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ddf39-6ebe-42b3-9d10-6c679764fb4f admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:18.352811 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ddf39-6ebe-42b3-9d10-6c679764fb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:18.353086 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611ddf39-6ebe-42b3-9d10-6c679764fb4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:18.353810 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ddf39-6ebe-42b3-9d10-6c679764fb4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:18.355399 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611ddf39-6ebe-42b3-9d10-6c679764fb4f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:18.366616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1358/2705] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:18.370590 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519116b-f6a0-4ed1-bc8f-384a0081461b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:18.381202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519116b-f6a0-4ed1-bc8f-384a0081461b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:18.385970 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba66302-3fda-4284-b4d1-8655d270aa56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:18.386628 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba66302-3fda-4284-b4d1-8655d270aa56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:18.386912 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba66302-3fda-4284-b4d1-8655d270aa56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:18.387217 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba66302-3fda-4284-b4d1-8655d270aa56 admin admin] PATH_INFO: `/v3/projects/ea5dbb8880454a6f9771230f33238149/users/345238d0f11b43cdb2c4fd964f75e87e/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:18.401956 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba66302-3fda-4284-b4d1-8655d270aa56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea5dbb8880454a6f9771230f33238149, user_id=345238d0f11b43cdb2c4fd964f75e87e, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:18.404587 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba66302-3fda-4284-b4d1-8655d270aa56 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:18.418678 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1348/2706] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:18 2022] PUT /identity/v3/projects/ea5dbb8880454a6f9771230f33238149/users/345238d0f11b43cdb2c4fd964f75e87e/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:18.431241 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91fc18-92e2-4ab8-9e11-805eab70a9eb None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:18.431625 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91fc18-92e2-4ab8-9e11-805eab70a9eb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:18.431802 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91fc18-92e2-4ab8-9e11-805eab70a9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:18.460103 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2f91fc18-92e2-4ab8-9e11-805eab70a9eb None None] MFA Rules not processed for user `345238d0f11b43cdb2c4fd964f75e87e`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:18.461250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2f91fc18-92e2-4ab8-9e11-805eab70a9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:18.531885 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1359/2707] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:18 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:18.537205 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba66302-3fda-4284-b4d1-8655d270aa56 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:18.598556 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba66302-3fda-4284-b4d1-8655d270aa56 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:18.603263 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7e32e-6b8b-4d26-b574-1a0dbfbfcb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:18.603870 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7e32e-6b8b-4d26-b574-1a0dbfbfcb3f admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:18.604152 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7e32e-6b8b-4d26-b574-1a0dbfbfcb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:18.604446 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7e32e-6b8b-4d26-b574-1a0dbfbfcb3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:18.605095 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7e32e-6b8b-4d26-b574-1a0dbfbfcb3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:18.607019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7e32e-6b8b-4d26-b574-1a0dbfbfcb3f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:18.634134 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1349/2708] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:18 2022] POST /identity/v3/projects => generated 409 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:18.638673 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91fc18-92e2-4ab8-9e11-805eab70a9eb None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:18.651425 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91fc18-92e2-4ab8-9e11-805eab70a9eb None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:18.706952 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b7175-dcd0-4dbc-8447-57b3433762f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:18.707558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b7175-dcd0-4dbc-8447-57b3433762f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:18.707835 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b7175-dcd0-4dbc-8447-57b3433762f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:18.708104 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531b7175-dcd0-4dbc-8447-57b3433762f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:18.708750 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b7175-dcd0-4dbc-8447-57b3433762f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:18.710842 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531b7175-dcd0-4dbc-8447-57b3433762f1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:18.731400 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1360/2709] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:18 2022] POST /identity/v3/users => generated 311 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:18.735477 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7e32e-6b8b-4d26-b574-1a0dbfbfcb3f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:18.747250 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7e32e-6b8b-4d26-b574-1a0dbfbfcb3f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:18.752420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75c222-59da-4e6b-b9ab-dfae48d3c6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:18.753724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75c222-59da-4e6b-b9ab-dfae48d3c6bb admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:18.754007 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75c222-59da-4e6b-b9ab-dfae48d3c6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:18.754280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd75c222-59da-4e6b-b9ab-dfae48d3c6bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:18.754977 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75c222-59da-4e6b-b9ab-dfae48d3c6bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:18.756580 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd75c222-59da-4e6b-b9ab-dfae48d3c6bb admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:18.767302 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1350/2710] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:18.771537 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b7175-dcd0-4dbc-8447-57b3433762f1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:18.781553 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531b7175-dcd0-4dbc-8447-57b3433762f1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:18.786269 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e9f59-a1d1-437f-87e0-a5faed4c948b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:18.786919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e9f59-a1d1-437f-87e0-a5faed4c948b admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:18.787201 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e9f59-a1d1-437f-87e0-a5faed4c948b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:18.787476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e9f59-a1d1-437f-87e0-a5faed4c948b admin admin] PATH_INFO: `/v3/projects/84dfbaf36f1e4b47afd70eab63bed1f2/users/25ddf9b1a34643de8a87185d19fd415c/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:18.802694 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e9f59-a1d1-437f-87e0-a5faed4c948b admin admin] RBAC: Authorizing `identity:create_grant(project_id=84dfbaf36f1e4b47afd70eab63bed1f2, user_id=25ddf9b1a34643de8a87185d19fd415c, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:18.805363 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e9f59-a1d1-437f-87e0-a5faed4c948b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:18.819964 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1361/2711] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:18 2022] PUT /identity/v3/projects/84dfbaf36f1e4b47afd70eab63bed1f2/users/25ddf9b1a34643de8a87185d19fd415c/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:18.823876 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75c222-59da-4e6b-b9ab-dfae48d3c6bb admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:18.836156 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd75c222-59da-4e6b-b9ab-dfae48d3c6bb admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:18.891037 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:18.898847 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df633b8-c234-40dc-88cc-35bacce1be30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:18.899483 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df633b8-c234-40dc-88cc-35bacce1be30 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:18.899917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df633b8-c234-40dc-88cc-35bacce1be30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:18.900344 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df633b8-c234-40dc-88cc-35bacce1be30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:18.901237 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df633b8-c234-40dc-88cc-35bacce1be30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:18.902861 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df633b8-c234-40dc-88cc-35bacce1be30 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:18.913229 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1351/2712] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:18 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:18.917312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e9f59-a1d1-437f-87e0-a5faed4c948b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:18.928606 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e9f59-a1d1-437f-87e0-a5faed4c948b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:18.933222 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba707895-f575-4592-a7e7-eb5363780ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:18.933974 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba707895-f575-4592-a7e7-eb5363780ffc admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:18.934259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba707895-f575-4592-a7e7-eb5363780ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:18.934536 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba707895-f575-4592-a7e7-eb5363780ffc admin admin] PATH_INFO: `/v3/projects/84dfbaf36f1e4b47afd70eab63bed1f2/users/25ddf9b1a34643de8a87185d19fd415c/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:18.936506 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba707895-f575-4592-a7e7-eb5363780ffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=84dfbaf36f1e4b47afd70eab63bed1f2, user_id=25ddf9b1a34643de8a87185d19fd415c, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:18.939208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba707895-f575-4592-a7e7-eb5363780ffc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:18.950085 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1362/2713] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:18 2022] PUT /identity/v3/projects/84dfbaf36f1e4b47afd70eab63bed1f2/users/25ddf9b1a34643de8a87185d19fd415c/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:18.960496 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3605dc7-871c-4460-b2ce-6689fcd8f95b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:18.960787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3605dc7-871c-4460-b2ce-6689fcd8f95b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:18.961132 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3605dc7-871c-4460-b2ce-6689fcd8f95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:18.992820 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-e3605dc7-871c-4460-b2ce-6689fcd8f95b None None] MFA Rules not processed for user `25ddf9b1a34643de8a87185d19fd415c`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:18.994062 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e3605dc7-871c-4460-b2ce-6689fcd8f95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:19.061019 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1352/2714] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:18 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:19.072262 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba707895-f575-4592-a7e7-eb5363780ffc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:19.074053 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ba707895-f575-4592-a7e7-eb5363780ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:19.081561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:19.138269 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba707895-f575-4592-a7e7-eb5363780ffc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:19.141953 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d21b9-7c6e-4488-b2af-9b1a19827ca4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:19.142570 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d21b9-7c6e-4488-b2af-9b1a19827ca4 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:19.142855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d21b9-7c6e-4488-b2af-9b1a19827ca4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:19.143521 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d21b9-7c6e-4488-b2af-9b1a19827ca4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:19.152793 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d21b9-7c6e-4488-b2af-9b1a19827ca4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:19.154342 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d21b9-7c6e-4488-b2af-9b1a19827ca4 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:19.164434 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1363/2715] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:19 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:19.323625 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3605dc7-871c-4460-b2ce-6689fcd8f95b None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:19.325093 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e3605dc7-871c-4460-b2ce-6689fcd8f95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:19.338511 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3605dc7-871c-4460-b2ce-6689fcd8f95b None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:19.342111 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e23955-5c70-4b13-96cf-fdc28043c418 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:19.342728 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e23955-5c70-4b13-96cf-fdc28043c418 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:19.343012 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e23955-5c70-4b13-96cf-fdc28043c418 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:19.343289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e23955-5c70-4b13-96cf-fdc28043c418 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:19.401057 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e23955-5c70-4b13-96cf-fdc28043c418 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:19.402519 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e23955-5c70-4b13-96cf-fdc28043c418 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:19.412235 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1353/2716] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:19 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:29.160167 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=24 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:29.407224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=66 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:31.473045 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d21b9-7c6e-4488-b2af-9b1a19827ca4 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:31.475211 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f76d21b9-7c6e-4488-b2af-9b1a19827ca4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:31.545455 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d21b9-7c6e-4488-b2af-9b1a19827ca4 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:31.551657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c29d8a-21f3-4bc7-975f-65c16565ba1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:31.552468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c29d8a-21f3-4bc7-975f-65c16565ba1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:31.552825 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c29d8a-21f3-4bc7-975f-65c16565ba1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:31.553172 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c29d8a-21f3-4bc7-975f-65c16565ba1d admin admin] PATH_INFO: `/v3/users/62fe654d142e4301a298c0a4b474e095` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:31.554651 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c29d8a-21f3-4bc7-975f-65c16565ba1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=62fe654d142e4301a298c0a4b474e095)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:31.556622 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c29d8a-21f3-4bc7-975f-65c16565ba1d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:31.603787 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f8c29d8a-21f3-4bc7-975f-65c16565ba1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62fe654d142e4301a298c0a4b474e095'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:31.610717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f8c29d8a-21f3-4bc7-975f-65c16565ba1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62fe654d142e4301a298c0a4b474e095'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:31.611381 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f8c29d8a-21f3-4bc7-975f-65c16565ba1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62fe654d142e4301a298c0a4b474e095'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:31.611985 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f8c29d8a-21f3-4bc7-975f-65c16565ba1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62fe654d142e4301a298c0a4b474e095'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:31.623853 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f8c29d8a-21f3-4bc7-975f-65c16565ba1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62fe654d142e4301a298c0a4b474e095'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:31.636721 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1364/2717] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:31 2022] DELETE /identity/v3/users/62fe654d142e4301a298c0a4b474e095 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:31.641463 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc85b5-7fbd-475c-ab1c-7f73679b0532 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:31.641996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc85b5-7fbd-475c-ab1c-7f73679b0532 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:31.642102 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc85b5-7fbd-475c-ab1c-7f73679b0532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:31.662041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-09bc85b5-7fbd-475c-ab1c-7f73679b0532 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:31.663279 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-09bc85b5-7fbd-475c-ab1c-7f73679b0532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:31.720359 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1354/2718] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:31.730784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c29d8a-21f3-4bc7-975f-65c16565ba1d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:31.732183 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f8c29d8a-21f3-4bc7-975f-65c16565ba1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:31.816717 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c29d8a-21f3-4bc7-975f-65c16565ba1d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:31.820726 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edd250-c093-4661-807a-359b6e8d30f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:31.821365 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edd250-c093-4661-807a-359b6e8d30f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:31.821658 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edd250-c093-4661-807a-359b6e8d30f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:31.821935 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edd250-c093-4661-807a-359b6e8d30f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:31.834811 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edd250-c093-4661-807a-359b6e8d30f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:31.834811 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edd250-c093-4661-807a-359b6e8d30f9 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:31.843286 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1365/2719] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:31.986944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc85b5-7fbd-475c-ab1c-7f73679b0532 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:31.989223 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-09bc85b5-7fbd-475c-ab1c-7f73679b0532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:32.001442 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc85b5-7fbd-475c-ab1c-7f73679b0532 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:32.007895 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a49d1c-b92b-4a40-8f5b-065edbd33645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:32.008616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a49d1c-b92b-4a40-8f5b-065edbd33645 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:32.008985 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a49d1c-b92b-4a40-8f5b-065edbd33645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:32.009325 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a49d1c-b92b-4a40-8f5b-065edbd33645 admin admin] PATH_INFO: `/v3/users/345238d0f11b43cdb2c4fd964f75e87e` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:32.010615 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a49d1c-b92b-4a40-8f5b-065edbd33645 admin admin] RBAC: Authorizing `identity:delete_user(user_id=345238d0f11b43cdb2c4fd964f75e87e)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:32.012615 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a49d1c-b92b-4a40-8f5b-065edbd33645 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:32.062860 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-29a49d1c-b92b-4a40-8f5b-065edbd33645 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '345238d0f11b43cdb2c4fd964f75e87e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:32.070702 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-29a49d1c-b92b-4a40-8f5b-065edbd33645 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '345238d0f11b43cdb2c4fd964f75e87e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:32.077965 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-29a49d1c-b92b-4a40-8f5b-065edbd33645 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '345238d0f11b43cdb2c4fd964f75e87e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:32.078622 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-29a49d1c-b92b-4a40-8f5b-065edbd33645 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '345238d0f11b43cdb2c4fd964f75e87e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:32.090714 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-29a49d1c-b92b-4a40-8f5b-065edbd33645 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '345238d0f11b43cdb2c4fd964f75e87e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:32.096715 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1355/2720] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:31 2022] DELETE /identity/v3/users/345238d0f11b43cdb2c4fd964f75e87e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:32.102369 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074169d-494f-411e-b662-53851e3a535a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:32.102776 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074169d-494f-411e-b662-53851e3a535a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:32.103104 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074169d-494f-411e-b662-53851e3a535a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:32.111128 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:32.126592 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a49d1c-b92b-4a40-8f5b-065edbd33645 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:32.128370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-29a49d1c-b92b-4a40-8f5b-065edbd33645 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:32.224178 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a49d1c-b92b-4a40-8f5b-065edbd33645 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:32.229211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a4b80-797f-4a5a-b100-3efcdce210ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:32.230045 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a4b80-797f-4a5a-b100-3efcdce210ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:32.230421 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a4b80-797f-4a5a-b100-3efcdce210ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:32.230710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a4b80-797f-4a5a-b100-3efcdce210ac admin admin] PATH_INFO: `/v3/projects/30408255d29a4c8cbf34be91128fd182` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:32.231981 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a4b80-797f-4a5a-b100-3efcdce210ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=30408255d29a4c8cbf34be91128fd182)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:32.233984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a4b80-797f-4a5a-b100-3efcdce210ac admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:32.254678 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:32.286359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3074169d-494f-411e-b662-53851e3a535a None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:32.288282 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3074169d-494f-411e-b662-53851e3a535a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:32.299611 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-213a4b80-797f-4a5a-b100-3efcdce210ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30408255d29a4c8cbf34be91128fd182', 'request_id': 'req-213a4b80-797f-4a5a-b100-3efcdce210ac'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:32.311809 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-213a4b80-797f-4a5a-b100-3efcdce210ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30408255d29a4c8cbf34be91128fd182', 'request_id': 'req-213a4b80-797f-4a5a-b100-3efcdce210ac'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:32.320074 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-213a4b80-797f-4a5a-b100-3efcdce210ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30408255d29a4c8cbf34be91128fd182', 'request_id': 'req-213a4b80-797f-4a5a-b100-3efcdce210ac'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:32.334775 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-213a4b80-797f-4a5a-b100-3efcdce210ac admin admin] The token cache is being invalidate because project 30408255d29a4c8cbf34be91128fd182 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:32.335085 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-213a4b80-797f-4a5a-b100-3efcdce210ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:32.335842 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-213a4b80-797f-4a5a-b100-3efcdce210ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:32.337433 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1356/2721] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:32 2022] DELETE /identity/v3/projects/30408255d29a4c8cbf34be91128fd182 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:32.343346 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787471c-03af-4ec1-8e76-30d4fb2216d6 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:32.343872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787471c-03af-4ec1-8e76-30d4fb2216d6 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:32.344243 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3787471c-03af-4ec1-8e76-30d4fb2216d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:32.372028 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3787471c-03af-4ec1-8e76-30d4fb2216d6 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:32.373669 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3787471c-03af-4ec1-8e76-30d4fb2216d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:32.378537 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1366/2722] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:32.384221 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1357/2723] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:36:32.390727 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074169d-494f-411e-b662-53851e3a535a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:32.392484 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b862d9-5987-404b-a8c1-24fd25108a32 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:32.393138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b862d9-5987-404b-a8c1-24fd25108a32 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:32.393138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b862d9-5987-404b-a8c1-24fd25108a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:32.398846 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3074169d-494f-411e-b662-53851e3a535a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:32.429065 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-33b862d9-5987-404b-a8c1-24fd25108a32 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:32.430780 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-33b862d9-5987-404b-a8c1-24fd25108a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:32.442186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1358/2724] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:32.447202 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b862d9-5987-404b-a8c1-24fd25108a32 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:32.462467 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b862d9-5987-404b-a8c1-24fd25108a32 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:32.470925 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516be68c-6d5d-4231-8d0d-67898efe8f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:32.471938 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516be68c-6d5d-4231-8d0d-67898efe8f98 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:32.472495 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516be68c-6d5d-4231-8d0d-67898efe8f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:32.473020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516be68c-6d5d-4231-8d0d-67898efe8f98 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:32.475969 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516be68c-6d5d-4231-8d0d-67898efe8f98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:36:32.476560 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516be68c-6d5d-4231-8d0d-67898efe8f98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:32.478678 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516be68c-6d5d-4231-8d0d-67898efe8f98 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:32.498874 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1359/2725] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:36:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:36:32.502969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074169d-494f-411e-b662-53851e3a535a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:32.504219 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516be68c-6d5d-4231-8d0d-67898efe8f98 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:32.508684 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02370946-d794-4eec-999f-0ab728c79c53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:32.509817 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02370946-d794-4eec-999f-0ab728c79c53 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:32.510403 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02370946-d794-4eec-999f-0ab728c79c53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:32.510850 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02370946-d794-4eec-999f-0ab728c79c53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:32.517518 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516be68c-6d5d-4231-8d0d-67898efe8f98 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:32.522950 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad3856-4891-4df1-a172-e4e9664ed077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:32.523580 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad3856-4891-4df1-a172-e4e9664ed077 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:32.523731 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02370946-d794-4eec-999f-0ab728c79c53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:32.523867 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad3856-4891-4df1-a172-e4e9664ed077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:32.524150 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dad3856-4891-4df1-a172-e4e9664ed077 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:32.524994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad3856-4891-4df1-a172-e4e9664ed077 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:32.525127 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02370946-d794-4eec-999f-0ab728c79c53 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:32.526933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dad3856-4891-4df1-a172-e4e9664ed077 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:32.535194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1367/2726] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:36:32.561199 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1360/2727] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:32 2022] POST /identity/v3/projects => generated 361 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:36:32.567640 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02370946-d794-4eec-999f-0ab728c79c53 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:32.582645 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02370946-d794-4eec-999f-0ab728c79c53 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:32.652851 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082f3ba-4adb-42f6-9baa-efc7c434c696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:32.653554 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082f3ba-4adb-42f6-9baa-efc7c434c696 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:32.653848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082f3ba-4adb-42f6-9baa-efc7c434c696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:32.654125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d082f3ba-4adb-42f6-9baa-efc7c434c696 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:32.654791 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d082f3ba-4adb-42f6-9baa-efc7c434c696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:32.656713 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d082f3ba-4adb-42f6-9baa-efc7c434c696 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:32.702959 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1368/2728] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:32 2022] POST /identity/v3/users => generated 281 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:32.707691 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad3856-4891-4df1-a172-e4e9664ed077 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:32.722971 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dad3856-4891-4df1-a172-e4e9664ed077 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:32.729107 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a017e6d-0803-413b-b1d4-e619c046e01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:32.730066 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a017e6d-0803-413b-b1d4-e619c046e01e admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:32.730346 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a017e6d-0803-413b-b1d4-e619c046e01e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:32.730616 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a017e6d-0803-413b-b1d4-e619c046e01e admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:32.731380 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a017e6d-0803-413b-b1d4-e619c046e01e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:32.733445 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a017e6d-0803-413b-b1d4-e619c046e01e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:32.745114 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1361/2729] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:32.756520 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082f3ba-4adb-42f6-9baa-efc7c434c696 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:32.768668 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d082f3ba-4adb-42f6-9baa-efc7c434c696 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:32.776619 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b004c6-0d74-4db5-abde-94a0c7cbe889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:32.782236 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b004c6-0d74-4db5-abde-94a0c7cbe889 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:32.782694 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b004c6-0d74-4db5-abde-94a0c7cbe889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:32.783151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b004c6-0d74-4db5-abde-94a0c7cbe889 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:32.784235 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b004c6-0d74-4db5-abde-94a0c7cbe889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:32.786250 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b004c6-0d74-4db5-abde-94a0c7cbe889 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:32.806132 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1369/2730] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:32 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:32.810318 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a017e6d-0803-413b-b1d4-e619c046e01e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:32.820738 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a017e6d-0803-413b-b1d4-e619c046e01e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:32.825449 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613b8be-326b-44fe-9ce6-5359e249de53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:32.826193 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613b8be-326b-44fe-9ce6-5359e249de53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:32.826474 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613b8be-326b-44fe-9ce6-5359e249de53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:32.826749 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613b8be-326b-44fe-9ce6-5359e249de53 admin admin] PATH_INFO: `/v3/projects/0a320872469c457ba7358099257945f7/users/bd1f34d009ad4789832d8120aadf50a6/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:32.842728 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613b8be-326b-44fe-9ce6-5359e249de53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a320872469c457ba7358099257945f7, user_id=bd1f34d009ad4789832d8120aadf50a6, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:32.845790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613b8be-326b-44fe-9ce6-5359e249de53 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:32.861903 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b004c6-0d74-4db5-abde-94a0c7cbe889 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:32.863894 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f4b004c6-0d74-4db5-abde-94a0c7cbe889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:32.870201 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1362/2731] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:32 2022] PUT /identity/v3/projects/0a320872469c457ba7358099257945f7/users/bd1f34d009ad4789832d8120aadf50a6/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:32.872051 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7bde3-7cdd-4a9e-bee9-4507ce399829 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:32.872411 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7bde3-7cdd-4a9e-bee9-4507ce399829 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:32.872752 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7bde3-7cdd-4a9e-bee9-4507ce399829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:32.885685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b004c6-0d74-4db5-abde-94a0c7cbe889 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:32.905045 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f5f7bde3-7cdd-4a9e-bee9-4507ce399829 None None] MFA Rules not processed for user `bd1f34d009ad4789832d8120aadf50a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:32.906322 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f5f7bde3-7cdd-4a9e-bee9-4507ce399829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:32.907900 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:32.950788 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5e2de-1324-4e34-b2d1-b05165507acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:32.951467 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5e2de-1324-4e34-b2d1-b05165507acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:32.951745 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5e2de-1324-4e34-b2d1-b05165507acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:32.952016 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5e2de-1324-4e34-b2d1-b05165507acb admin admin] PATH_INFO: `/v3/projects/ea5dbb8880454a6f9771230f33238149` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:32.953178 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5e2de-1324-4e34-b2d1-b05165507acb admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea5dbb8880454a6f9771230f33238149)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:32.955360 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5e2de-1324-4e34-b2d1-b05165507acb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:32.976781 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1363/2732] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:33.005663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4ec5e2de-1324-4e34-b2d1-b05165507acb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea5dbb8880454a6f9771230f33238149', 'request_id': 'req-4ec5e2de-1324-4e34-b2d1-b05165507acb'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:33.012069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4ec5e2de-1324-4e34-b2d1-b05165507acb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea5dbb8880454a6f9771230f33238149', 'request_id': 'req-4ec5e2de-1324-4e34-b2d1-b05165507acb'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:33.012541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7bde3-7cdd-4a9e-bee9-4507ce399829 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:33.014226 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f5f7bde3-7cdd-4a9e-bee9-4507ce399829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:33.017994 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4ec5e2de-1324-4e34-b2d1-b05165507acb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea5dbb8880454a6f9771230f33238149', 'request_id': 'req-4ec5e2de-1324-4e34-b2d1-b05165507acb'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:33.027878 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:33.032193 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4ec5e2de-1324-4e34-b2d1-b05165507acb admin admin] The token cache is being invalidate because project ea5dbb8880454a6f9771230f33238149 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:33.032488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4ec5e2de-1324-4e34-b2d1-b05165507acb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:33.033216 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4ec5e2de-1324-4e34-b2d1-b05165507acb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:33.036076 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1370/2733] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:32 2022] DELETE /identity/v3/projects/ea5dbb8880454a6f9771230f33238149 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:33.040024 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5e2de-1324-4e34-b2d1-b05165507acb admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:33.041399 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4ec5e2de-1324-4e34-b2d1-b05165507acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:33.098574 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7bde3-7cdd-4a9e-bee9-4507ce399829 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:33.105906 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5e2de-1324-4e34-b2d1-b05165507acb admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:33.110485 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e681c8-e006-4159-9e6d-314eede9eee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:33.111109 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e681c8-e006-4159-9e6d-314eede9eee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:33.111385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e681c8-e006-4159-9e6d-314eede9eee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:33.111659 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e681c8-e006-4159-9e6d-314eede9eee7 admin admin] PATH_INFO: `/v3/users/25ddf9b1a34643de8a87185d19fd415c` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:33.113460 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e681c8-e006-4159-9e6d-314eede9eee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25ddf9b1a34643de8a87185d19fd415c)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:33.115393 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e681c8-e006-4159-9e6d-314eede9eee7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:33.155042 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dd1186-5b45-4b71-ab9d-ae152429ddf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:33.156038 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd1186-5b45-4b71-ab9d-ae152429ddf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:33.156319 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd1186-5b45-4b71-ab9d-ae152429ddf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:33.156585 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dd1186-5b45-4b71-ab9d-ae152429ddf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:33.159456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-14dd1186-5b45-4b71-ab9d-ae152429ddf2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:33.166154 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-59e681c8-e006-4159-9e6d-314eede9eee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25ddf9b1a34643de8a87185d19fd415c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:33.174831 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-59e681c8-e006-4159-9e6d-314eede9eee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25ddf9b1a34643de8a87185d19fd415c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:33.176039 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-59e681c8-e006-4159-9e6d-314eede9eee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25ddf9b1a34643de8a87185d19fd415c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:33.176720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-59e681c8-e006-4159-9e6d-314eede9eee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25ddf9b1a34643de8a87185d19fd415c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:33.177878 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:33.191640 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-59e681c8-e006-4159-9e6d-314eede9eee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25ddf9b1a34643de8a87185d19fd415c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:33.204510 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1371/2734] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:33 2022] DELETE /identity/v3/users/25ddf9b1a34643de8a87185d19fd415c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:33.224420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd1186-5b45-4b71-ab9d-ae152429ddf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:33.226477 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dd1186-5b45-4b71-ab9d-ae152429ddf2 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:33.239311 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1364/2735] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:36:33.259437 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e681c8-e006-4159-9e6d-314eede9eee7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:33.262029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-59e681c8-e006-4159-9e6d-314eede9eee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:33.345916 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e681c8-e006-4159-9e6d-314eede9eee7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:33.351301 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706f368-0caf-455d-9c80-d9ab71556ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:33.351940 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706f368-0caf-455d-9c80-d9ab71556ca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:33.352217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706f368-0caf-455d-9c80-d9ab71556ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:33.352487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706f368-0caf-455d-9c80-d9ab71556ca3 admin admin] PATH_INFO: `/v3/users/bd1f34d009ad4789832d8120aadf50a6` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:33.353741 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706f368-0caf-455d-9c80-d9ab71556ca3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd1f34d009ad4789832d8120aadf50a6)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:33.355663 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706f368-0caf-455d-9c80-d9ab71556ca3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:33.407921 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f706f368-0caf-455d-9c80-d9ab71556ca3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd1f34d009ad4789832d8120aadf50a6'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:33.415803 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f706f368-0caf-455d-9c80-d9ab71556ca3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd1f34d009ad4789832d8120aadf50a6'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:33.416498 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f706f368-0caf-455d-9c80-d9ab71556ca3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd1f34d009ad4789832d8120aadf50a6'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:33.417125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f706f368-0caf-455d-9c80-d9ab71556ca3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd1f34d009ad4789832d8120aadf50a6'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:33.429387 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f706f368-0caf-455d-9c80-d9ab71556ca3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd1f34d009ad4789832d8120aadf50a6'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:33.443062 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1372/2736] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:33 2022] DELETE /identity/v3/users/bd1f34d009ad4789832d8120aadf50a6 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:33.447407 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd789e41-2a7c-4fe4-8175-e65fd5a39953 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:33.447704 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd789e41-2a7c-4fe4-8175-e65fd5a39953 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:33.447987 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd789e41-2a7c-4fe4-8175-e65fd5a39953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:33.468898 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-dd789e41-2a7c-4fe4-8175-e65fd5a39953 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:33.470652 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-dd789e41-2a7c-4fe4-8175-e65fd5a39953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:33.494032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706f368-0caf-455d-9c80-d9ab71556ca3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:33.495759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f706f368-0caf-455d-9c80-d9ab71556ca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:33.534133 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:33.543462 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1365/2737] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:33.552888 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:33.553399 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd789e41-2a7c-4fe4-8175-e65fd5a39953 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:33.554936 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-dd789e41-2a7c-4fe4-8175-e65fd5a39953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:33.568094 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706f368-0caf-455d-9c80-d9ab71556ca3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:33.573196 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a1d82-1f6d-4d4c-a590-6cd6d1304855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:33.573866 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1d82-1f6d-4d4c-a590-6cd6d1304855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:33.574146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1d82-1f6d-4d4c-a590-6cd6d1304855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:33.574422 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a1d82-1f6d-4d4c-a590-6cd6d1304855 admin admin] PATH_INFO: `/v3/projects/84dfbaf36f1e4b47afd70eab63bed1f2` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:33.575591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a1d82-1f6d-4d4c-a590-6cd6d1304855 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84dfbaf36f1e4b47afd70eab63bed1f2)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:33.577478 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a1d82-1f6d-4d4c-a590-6cd6d1304855 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:33.614552 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd789e41-2a7c-4fe4-8175-e65fd5a39953 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:33.624038 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ba5a1d82-1f6d-4d4c-a590-6cd6d1304855 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84dfbaf36f1e4b47afd70eab63bed1f2', 'request_id': 'req-ba5a1d82-1f6d-4d4c-a590-6cd6d1304855'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:33.630584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ba5a1d82-1f6d-4d4c-a590-6cd6d1304855 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84dfbaf36f1e4b47afd70eab63bed1f2', 'request_id': 'req-ba5a1d82-1f6d-4d4c-a590-6cd6d1304855'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:33.636358 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ba5a1d82-1f6d-4d4c-a590-6cd6d1304855 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84dfbaf36f1e4b47afd70eab63bed1f2', 'request_id': 'req-ba5a1d82-1f6d-4d4c-a590-6cd6d1304855'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:33.650861 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ba5a1d82-1f6d-4d4c-a590-6cd6d1304855 admin admin] The token cache is being invalidate because project 84dfbaf36f1e4b47afd70eab63bed1f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:33.651154 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ba5a1d82-1f6d-4d4c-a590-6cd6d1304855 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:33.651828 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ba5a1d82-1f6d-4d4c-a590-6cd6d1304855 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:33.653349 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1373/2738] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:33 2022] DELETE /identity/v3/projects/84dfbaf36f1e4b47afd70eab63bed1f2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:33.658124 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96df7c-3b41-43f5-ab1c-6cf31addb454 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:33.658410 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96df7c-3b41-43f5-ab1c-6cf31addb454 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:33.658687 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96df7c-3b41-43f5-ab1c-6cf31addb454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:33.667675 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2a959-41d8-4746-85ef-9dd20fc67aa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:33.668300 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2a959-41d8-4746-85ef-9dd20fc67aa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:33.668586 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2a959-41d8-4746-85ef-9dd20fc67aa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:33.668866 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2a959-41d8-4746-85ef-9dd20fc67aa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:33.670657 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b4a2a959-41d8-4746-85ef-9dd20fc67aa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:33.679891 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3d96df7c-3b41-43f5-ab1c-6cf31addb454 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:33.681076 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3d96df7c-3b41-43f5-ab1c-6cf31addb454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:33.734765 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2a959-41d8-4746-85ef-9dd20fc67aa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:33.736210 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a2a959-41d8-4746-85ef-9dd20fc67aa5 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:33.747055 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1374/2739] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:36:33.747229 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1366/2740] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:36:33.749634 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa2561-cfa5-42c3-b521-aa3e83674711 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:33.749949 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa2561-cfa5-42c3-b521-aa3e83674711 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:33.750247 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaa2561-cfa5-42c3-b521-aa3e83674711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:33.772962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-feaa2561-cfa5-42c3-b521-aa3e83674711 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:33.774357 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-feaa2561-cfa5-42c3-b521-aa3e83674711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:33.782724 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1375/2741] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:36:33.787842 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2a959-41d8-4746-85ef-9dd20fc67aa5 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:33.798198 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a2a959-41d8-4746-85ef-9dd20fc67aa5 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:33.803997 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3879b83-e58c-45b3-a3f2-2744e5cd2c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:33.804724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3879b83-e58c-45b3-a3f2-2744e5cd2c2a admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:33.805067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3879b83-e58c-45b3-a3f2-2744e5cd2c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:33.805492 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3879b83-e58c-45b3-a3f2-2744e5cd2c2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:33.806519 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3879b83-e58c-45b3-a3f2-2744e5cd2c2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:36:33.806519 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3879b83-e58c-45b3-a3f2-2744e5cd2c2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:33.808176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3879b83-e58c-45b3-a3f2-2744e5cd2c2a admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:33.822540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1367/2742] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:36:33.826886 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa2561-cfa5-42c3-b521-aa3e83674711 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:33.838415 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaa2561-cfa5-42c3-b521-aa3e83674711 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:33.843330 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25372ac7-0715-4f74-801d-abe6cf1d5374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:33.843957 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25372ac7-0715-4f74-801d-abe6cf1d5374 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:33.844300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25372ac7-0715-4f74-801d-abe6cf1d5374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:33.844576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25372ac7-0715-4f74-801d-abe6cf1d5374 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:33.845239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25372ac7-0715-4f74-801d-abe6cf1d5374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:33.847346 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25372ac7-0715-4f74-801d-abe6cf1d5374 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:33.884605 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1376/2743] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:33 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:33.888920 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3879b83-e58c-45b3-a3f2-2744e5cd2c2a admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:33.903226 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3879b83-e58c-45b3-a3f2-2744e5cd2c2a admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:33.965362 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:33.978393 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b44a14-d9ff-487c-86ee-ff40ccc4d45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:33.978943 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b44a14-d9ff-487c-86ee-ff40ccc4d45f admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:33.979361 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b44a14-d9ff-487c-86ee-ff40ccc4d45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:33.979721 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b44a14-d9ff-487c-86ee-ff40ccc4d45f admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:33.980600 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b44a14-d9ff-487c-86ee-ff40ccc4d45f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:33.982486 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b44a14-d9ff-487c-86ee-ff40ccc4d45f admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:34.007548 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1368/2744] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:33 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:34.011591 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25372ac7-0715-4f74-801d-abe6cf1d5374 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:34.023156 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25372ac7-0715-4f74-801d-abe6cf1d5374 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:34.028494 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5dc1c-172d-4d39-801e-6938ddae314b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:34.029335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5dc1c-172d-4d39-801e-6938ddae314b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:34.029847 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5dc1c-172d-4d39-801e-6938ddae314b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:34.030345 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5dc1c-172d-4d39-801e-6938ddae314b admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:34.031616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5dc1c-172d-4d39-801e-6938ddae314b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:34.034589 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5dc1c-172d-4d39-801e-6938ddae314b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:34.051057 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1377/2745] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:34.057644 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b44a14-d9ff-487c-86ee-ff40ccc4d45f admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:34.066849 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5dc1c-172d-4d39-801e-6938ddae314b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:34.069907 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-42a5dc1c-172d-4d39-801e-6938ddae314b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:34.072295 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b44a14-d9ff-487c-86ee-ff40ccc4d45f admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:34.079256 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea9f19-bace-4cb5-b364-44ee01355557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:34.080364 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea9f19-bace-4cb5-b364-44ee01355557 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:34.080670 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea9f19-bace-4cb5-b364-44ee01355557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:34.080931 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea9f19-bace-4cb5-b364-44ee01355557 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:34.082211 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea9f19-bace-4cb5-b364-44ee01355557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:34.083959 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea9f19-bace-4cb5-b364-44ee01355557 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:34.084841 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5dc1c-172d-4d39-801e-6938ddae314b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:34.090371 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c524055-8f7f-436c-a6c0-5f75faf5b289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:34.091069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c524055-8f7f-436c-a6c0-5f75faf5b289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:34.091242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c524055-8f7f-436c-a6c0-5f75faf5b289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:34.091505 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c524055-8f7f-436c-a6c0-5f75faf5b289 admin admin] PATH_INFO: `/v3/projects/0a320872469c457ba7358099257945f7` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:34.099161 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c524055-8f7f-436c-a6c0-5f75faf5b289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a320872469c457ba7358099257945f7)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:34.101091 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c524055-8f7f-436c-a6c0-5f75faf5b289 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:34.103949 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1369/2746] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:34.108377 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea9f19-bace-4cb5-b364-44ee01355557 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:34.118841 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea9f19-bace-4cb5-b364-44ee01355557 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:34.129513 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8273e5f-7bb3-445b-8bd4-031dde752495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:34.130275 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8273e5f-7bb3-445b-8bd4-031dde752495 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:34.130562 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8273e5f-7bb3-445b-8bd4-031dde752495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:34.130840 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8273e5f-7bb3-445b-8bd4-031dde752495 admin admin] PATH_INFO: `/v3/projects/9510a6389ca84ba88ccc25a88133af22/users/5b7695f708414c1395e6c37374ba78ad/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:34.159358 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8273e5f-7bb3-445b-8bd4-031dde752495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9510a6389ca84ba88ccc25a88133af22, user_id=5b7695f708414c1395e6c37374ba78ad, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:34.163518 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8273e5f-7bb3-445b-8bd4-031dde752495 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:34.172269 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:34.180875 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5c524055-8f7f-436c-a6c0-5f75faf5b289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a320872469c457ba7358099257945f7', 'request_id': 'req-5c524055-8f7f-436c-a6c0-5f75faf5b289'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:34.184132 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1370/2747] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:34 2022] PUT /identity/v3/projects/9510a6389ca84ba88ccc25a88133af22/users/5b7695f708414c1395e6c37374ba78ad/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:34.189441 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5c524055-8f7f-436c-a6c0-5f75faf5b289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a320872469c457ba7358099257945f7', 'request_id': 'req-5c524055-8f7f-436c-a6c0-5f75faf5b289'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:34.195963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5c524055-8f7f-436c-a6c0-5f75faf5b289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a320872469c457ba7358099257945f7', 'request_id': 'req-5c524055-8f7f-436c-a6c0-5f75faf5b289'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:34.202390 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d2485-364a-458b-905c-c494da35749a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:34.203113 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d2485-364a-458b-905c-c494da35749a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:34.203484 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d2485-364a-458b-905c-c494da35749a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:34.213123 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5c524055-8f7f-436c-a6c0-5f75faf5b289 admin admin] The token cache is being invalidate because project 0a320872469c457ba7358099257945f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:34.213307 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5c524055-8f7f-436c-a6c0-5f75faf5b289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:34.214008 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5c524055-8f7f-436c-a6c0-5f75faf5b289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:34.215866 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1378/2748] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:34 2022] DELETE /identity/v3/projects/0a320872469c457ba7358099257945f7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:34.221288 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2bde5-3d9a-438e-8cae-caa612e4b854 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:34.221626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2bde5-3d9a-438e-8cae-caa612e4b854 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:34.221909 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2bde5-3d9a-438e-8cae-caa612e4b854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:34.235907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-745d2485-364a-458b-905c-c494da35749a None None] MFA Rules not processed for user `5b7695f708414c1395e6c37374ba78ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:34.237451 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-745d2485-364a-458b-905c-c494da35749a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:34.243543 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f2e2bde5-3d9a-438e-8cae-caa612e4b854 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:34.244743 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f2e2bde5-3d9a-438e-8cae-caa612e4b854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:34.310507 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1379/2749] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:34.319227 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1371/2750] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:34.323889 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff90ea-daa2-4369-81dd-6abdec4c32ac None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:34.324184 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff90ea-daa2-4369-81dd-6abdec4c32ac None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:34.324469 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff90ea-daa2-4369-81dd-6abdec4c32ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:34.332158 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d2485-364a-458b-905c-c494da35749a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:34.333846 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-745d2485-364a-458b-905c-c494da35749a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:34.350292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-abff90ea-daa2-4369-81dd-6abdec4c32ac None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:34.351588 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-abff90ea-daa2-4369-81dd-6abdec4c32ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:34.359740 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1380/2751] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:36:34.364423 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff90ea-daa2-4369-81dd-6abdec4c32ac None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:34.373917 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff90ea-daa2-4369-81dd-6abdec4c32ac None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:34.379934 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb355138-5e27-4888-b5b0-33e56a5e119b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:34.380629 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb355138-5e27-4888-b5b0-33e56a5e119b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:34.380914 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb355138-5e27-4888-b5b0-33e56a5e119b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:34.381187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb355138-5e27-4888-b5b0-33e56a5e119b admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:34.382041 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb355138-5e27-4888-b5b0-33e56a5e119b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:36:34.382345 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb355138-5e27-4888-b5b0-33e56a5e119b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:34.384001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb355138-5e27-4888-b5b0-33e56a5e119b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:34.395923 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d2485-364a-458b-905c-c494da35749a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:34.397048 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1381/2752] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:36:34.399657 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4026a2-78ed-408d-ad0a-c81eaf7cd2bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:34.400278 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4026a2-78ed-408d-ad0a-c81eaf7cd2bb service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:34.400798 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4026a2-78ed-408d-ad0a-c81eaf7cd2bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:34.400929 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4026a2-78ed-408d-ad0a-c81eaf7cd2bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:34.401669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb355138-5e27-4888-b5b0-33e56a5e119b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:34.409735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4026a2-78ed-408d-ad0a-c81eaf7cd2bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:34.411530 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb355138-5e27-4888-b5b0-33e56a5e119b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:34.411760 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4026a2-78ed-408d-ad0a-c81eaf7cd2bb service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:34.416183 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448e44f-470d-441b-9ce9-e2930ffde04c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:34.416765 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448e44f-470d-441b-9ce9-e2930ffde04c admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:34.417044 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448e44f-470d-441b-9ce9-e2930ffde04c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:34.417313 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e448e44f-470d-441b-9ce9-e2930ffde04c admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:34.417987 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448e44f-470d-441b-9ce9-e2930ffde04c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:34.419847 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e448e44f-470d-441b-9ce9-e2930ffde04c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:34.420671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1372/2753] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:36:34.439643 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4026a2-78ed-408d-ad0a-c81eaf7cd2bb service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:34.442409 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1a4026a2-78ed-408d-ad0a-c81eaf7cd2bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:34.452148 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1382/2754] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:34 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:36:34.456380 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448e44f-470d-441b-9ce9-e2930ffde04c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:34.458597 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4026a2-78ed-408d-ad0a-c81eaf7cd2bb service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:34.498073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:34.517981 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc04738-051c-4ffa-b501-e64f1003bfad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:34.518630 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc04738-051c-4ffa-b501-e64f1003bfad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:34.519437 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc04738-051c-4ffa-b501-e64f1003bfad admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:34.519704 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc04738-051c-4ffa-b501-e64f1003bfad admin admin] PATH_INFO: `/v3/users/5b7695f708414c1395e6c37374ba78ad` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:34.520917 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc04738-051c-4ffa-b501-e64f1003bfad admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b7695f708414c1395e6c37374ba78ad)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:34.521072 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e448e44f-470d-441b-9ce9-e2930ffde04c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:34.522881 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc04738-051c-4ffa-b501-e64f1003bfad admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:34.525785 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eecbad-0ae2-498e-8706-53c802bbb00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:34.526515 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eecbad-0ae2-498e-8706-53c802bbb00f admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:34.526802 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eecbad-0ae2-498e-8706-53c802bbb00f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:34.527075 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eecbad-0ae2-498e-8706-53c802bbb00f admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:34.527742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eecbad-0ae2-498e-8706-53c802bbb00f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:34.529679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eecbad-0ae2-498e-8706-53c802bbb00f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:34.553411 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1383/2755] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:34 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:36:34.557507 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eecbad-0ae2-498e-8706-53c802bbb00f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:34.569090 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eecbad-0ae2-498e-8706-53c802bbb00f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:34.574712 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfc84f-c2fd-4970-add1-053e626caef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:34.575371 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfc84f-c2fd-4970-add1-053e626caef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:34.575651 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfc84f-c2fd-4970-add1-053e626caef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:34.575941 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfc84f-c2fd-4970-add1-053e626caef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:34.576673 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfc84f-c2fd-4970-add1-053e626caef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:34.578325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfc84f-c2fd-4970-add1-053e626caef3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:34.580303 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9dc04738-051c-4ffa-b501-e64f1003bfad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b7695f708414c1395e6c37374ba78ad'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:34.590751 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9dc04738-051c-4ffa-b501-e64f1003bfad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b7695f708414c1395e6c37374ba78ad'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:34.592818 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1384/2756] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:36:34.596547 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfc84f-c2fd-4970-add1-053e626caef3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:34.600728 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9dc04738-051c-4ffa-b501-e64f1003bfad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b7695f708414c1395e6c37374ba78ad'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:34.601434 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9dc04738-051c-4ffa-b501-e64f1003bfad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b7695f708414c1395e6c37374ba78ad'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:34.614046 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:34.615410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-9dc04738-051c-4ffa-b501-e64f1003bfad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b7695f708414c1395e6c37374ba78ad'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:34.618904 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfc84f-c2fd-4970-add1-053e626caef3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:34.622201 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1373/2757] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:34 2022] DELETE /identity/v3/users/5b7695f708414c1395e6c37374ba78ad => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:34.624536 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3578fe2-0d40-400a-ab32-e0590dd02281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:34.625165 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3578fe2-0d40-400a-ab32-e0590dd02281 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:34.625629 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3578fe2-0d40-400a-ab32-e0590dd02281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:34.626330 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3578fe2-0d40-400a-ab32-e0590dd02281 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:34.627008 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210f2f4-04c7-4151-b20a-a76c6b2191fd None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:34.627262 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210f2f4-04c7-4151-b20a-a76c6b2191fd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:34.627780 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3578fe2-0d40-400a-ab32-e0590dd02281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:34.627904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210f2f4-04c7-4151-b20a-a76c6b2191fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:34.630243 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3578fe2-0d40-400a-ab32-e0590dd02281 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:34.641578 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1385/2758] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:34.646109 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3578fe2-0d40-400a-ab32-e0590dd02281 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:34.647901 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e3578fe2-0d40-400a-ab32-e0590dd02281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:34.648027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3210f2f4-04c7-4151-b20a-a76c6b2191fd None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:34.649155 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3210f2f4-04c7-4151-b20a-a76c6b2191fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:34.661036 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3578fe2-0d40-400a-ab32-e0590dd02281 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:34.710849 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1374/2759] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:36:34.720155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a9b72-4c3f-46a7-b626-4105603cc0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:34.720331 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210f2f4-04c7-4151-b20a-a76c6b2191fd None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:34.720817 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a9b72-4c3f-46a7-b626-4105603cc0ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:34.721096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a9b72-4c3f-46a7-b626-4105603cc0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:34.721367 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a9b72-4c3f-46a7-b626-4105603cc0ca admin admin] PATH_INFO: `/v3/projects/8c94b7f889f847a1aee252d7e7583961/users/5c5b5142c8bf49f58b94defd51697694/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:34.721631 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3210f2f4-04c7-4151-b20a-a76c6b2191fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:34.736026 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a9b72-4c3f-46a7-b626-4105603cc0ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c94b7f889f847a1aee252d7e7583961, user_id=5c5b5142c8bf49f58b94defd51697694, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:34.739508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a9b72-4c3f-46a7-b626-4105603cc0ca admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:34.752897 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1386/2760] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:34 2022] PUT /identity/v3/projects/8c94b7f889f847a1aee252d7e7583961/users/5c5b5142c8bf49f58b94defd51697694/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:34.782727 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210f2f4-04c7-4151-b20a-a76c6b2191fd None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:34.797295 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:34.822476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6680f9-024c-4b02-a2ad-78461d0b380b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:34.822677 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6680f9-024c-4b02-a2ad-78461d0b380b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:34.822937 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6680f9-024c-4b02-a2ad-78461d0b380b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:34.840036 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399572e7-c1ac-4270-b082-5168cd843da2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:34.840653 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399572e7-c1ac-4270-b082-5168cd843da2 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:34.840933 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399572e7-c1ac-4270-b082-5168cd843da2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:34.841205 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399572e7-c1ac-4270-b082-5168cd843da2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:34.850996 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ca6680f9-024c-4b02-a2ad-78461d0b380b None None] MFA Rules not processed for user `5c5b5142c8bf49f58b94defd51697694`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:34.852208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ca6680f9-024c-4b02-a2ad-78461d0b380b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:34.901845 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399572e7-c1ac-4270-b082-5168cd843da2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:34.903287 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399572e7-c1ac-4270-b082-5168cd843da2 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:34.914402 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1375/2761] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:36:34.916385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1387/2762] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:36:34.927398 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399572e7-c1ac-4270-b082-5168cd843da2 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:34.928785 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-399572e7-c1ac-4270-b082-5168cd843da2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:34.988002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399572e7-c1ac-4270-b082-5168cd843da2 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:34.994803 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2b5b90-3468-46af-bc54-b02c80be48fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:34.995610 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b5b90-3468-46af-bc54-b02c80be48fe service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:34.995824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b5b90-3468-46af-bc54-b02c80be48fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:34.996410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2b5b90-3468-46af-bc54-b02c80be48fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:35.007244 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b5b90-3468-46af-bc54-b02c80be48fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:35.008690 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2b5b90-3468-46af-bc54-b02c80be48fe service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:35.020790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1376/2763] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:35.212673 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6680f9-024c-4b02-a2ad-78461d0b380b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:35.214223 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ca6680f9-024c-4b02-a2ad-78461d0b380b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:35.232723 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6680f9-024c-4b02-a2ad-78461d0b380b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:35.264962 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d1add-3daf-469c-812e-cde230feba13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:35.265880 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d1add-3daf-469c-812e-cde230feba13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:35.266524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d1add-3daf-469c-812e-cde230feba13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:35.266807 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d1add-3daf-469c-812e-cde230feba13 admin admin] PATH_INFO: `/v3/projects/9510a6389ca84ba88ccc25a88133af22` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:35.268433 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d1add-3daf-469c-812e-cde230feba13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9510a6389ca84ba88ccc25a88133af22)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:35.270566 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d1add-3daf-469c-812e-cde230feba13 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:35.328937 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6b3d1add-3daf-469c-812e-cde230feba13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9510a6389ca84ba88ccc25a88133af22', 'request_id': 'req-6b3d1add-3daf-469c-812e-cde230feba13'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:35.336296 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6b3d1add-3daf-469c-812e-cde230feba13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9510a6389ca84ba88ccc25a88133af22', 'request_id': 'req-6b3d1add-3daf-469c-812e-cde230feba13'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:35.343140 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:35.348419 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6b3d1add-3daf-469c-812e-cde230feba13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9510a6389ca84ba88ccc25a88133af22', 'request_id': 'req-6b3d1add-3daf-469c-812e-cde230feba13'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:35.376690 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6b3d1add-3daf-469c-812e-cde230feba13 admin admin] The token cache is being invalidate because project 9510a6389ca84ba88ccc25a88133af22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:35.376980 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6b3d1add-3daf-469c-812e-cde230feba13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:35.377889 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6b3d1add-3daf-469c-812e-cde230feba13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:35.380140 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1388/2764] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:35 2022] DELETE /identity/v3/projects/9510a6389ca84ba88ccc25a88133af22 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:35.385030 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87163478-ba56-405f-bd44-8310b3c9956c None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:35.385321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87163478-ba56-405f-bd44-8310b3c9956c None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:35.385734 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87163478-ba56-405f-bd44-8310b3c9956c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:35.411227 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-87163478-ba56-405f-bd44-8310b3c9956c None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:35.412584 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-87163478-ba56-405f-bd44-8310b3c9956c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:35.476067 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1377/2765] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:36:35.484621 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35231d-e89c-4fca-894c-86b1da9b5fca None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:35.484908 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35231d-e89c-4fca-894c-86b1da9b5fca None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:35.485184 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35231d-e89c-4fca-894c-86b1da9b5fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:35.506045 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5c35231d-e89c-4fca-894c-86b1da9b5fca None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:35.507227 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5c35231d-e89c-4fca-894c-86b1da9b5fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:35.514943 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1389/2766] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:35.519045 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87163478-ba56-405f-bd44-8310b3c9956c None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:35.528788 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87163478-ba56-405f-bd44-8310b3c9956c None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:35.533367 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523f048-fe0f-491b-aafb-a3e3a4d4ca76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:35.534004 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523f048-fe0f-491b-aafb-a3e3a4d4ca76 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:35.534281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523f048-fe0f-491b-aafb-a3e3a4d4ca76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:35.534556 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523f048-fe0f-491b-aafb-a3e3a4d4ca76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:35.535475 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523f048-fe0f-491b-aafb-a3e3a4d4ca76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:36:35.535776 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523f048-fe0f-491b-aafb-a3e3a4d4ca76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:35.537356 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523f048-fe0f-491b-aafb-a3e3a4d4ca76 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:35.539704 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:35.549634 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1378/2767] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:36:35.554404 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35231d-e89c-4fca-894c-86b1da9b5fca None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:35.563599 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35231d-e89c-4fca-894c-86b1da9b5fca None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:35.569009 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e079d8-e8b5-437e-a9dc-e9606eaa7005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:35.569649 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e079d8-e8b5-437e-a9dc-e9606eaa7005 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:35.569934 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e079d8-e8b5-437e-a9dc-e9606eaa7005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:35.570236 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e079d8-e8b5-437e-a9dc-e9606eaa7005 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:35.570925 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e079d8-e8b5-437e-a9dc-e9606eaa7005 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:35.572813 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e079d8-e8b5-437e-a9dc-e9606eaa7005 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:35.599847 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1390/2768] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:35 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:35.603882 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523f048-fe0f-491b-aafb-a3e3a4d4ca76 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:35.614865 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523f048-fe0f-491b-aafb-a3e3a4d4ca76 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:35.671401 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49133489-d306-4ca5-8832-47cf40b638d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:35.672015 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49133489-d306-4ca5-8832-47cf40b638d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:35.672299 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49133489-d306-4ca5-8832-47cf40b638d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:35.672576 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49133489-d306-4ca5-8832-47cf40b638d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:35.673242 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49133489-d306-4ca5-8832-47cf40b638d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:35.675866 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49133489-d306-4ca5-8832-47cf40b638d1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:35.696826 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1379/2769] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:35 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:35.700868 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e079d8-e8b5-437e-a9dc-e9606eaa7005 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:35.711199 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e079d8-e8b5-437e-a9dc-e9606eaa7005 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:35.715686 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5c4c5-2b58-4a77-9899-8d3f4bb2ea2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:35.716281 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5c4c5-2b58-4a77-9899-8d3f4bb2ea2a admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:35.716562 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5c4c5-2b58-4a77-9899-8d3f4bb2ea2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:35.716828 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a5c4c5-2b58-4a77-9899-8d3f4bb2ea2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:35.717514 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5c4c5-2b58-4a77-9899-8d3f4bb2ea2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:35.719115 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a5c4c5-2b58-4a77-9899-8d3f4bb2ea2a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:35.729648 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1391/2770] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:35.733382 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49133489-d306-4ca5-8832-47cf40b638d1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:35.743951 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49133489-d306-4ca5-8832-47cf40b638d1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:35.748572 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee695b33-9e7b-453e-b5ec-54a30cc59ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:35.749173 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee695b33-9e7b-453e-b5ec-54a30cc59ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:35.749458 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee695b33-9e7b-453e-b5ec-54a30cc59ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:35.749770 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee695b33-9e7b-453e-b5ec-54a30cc59ff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:35.750457 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee695b33-9e7b-453e-b5ec-54a30cc59ff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:35.752082 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee695b33-9e7b-453e-b5ec-54a30cc59ff4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:35.762724 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1380/2771] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:35.766698 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5c4c5-2b58-4a77-9899-8d3f4bb2ea2a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:35.776539 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a5c4c5-2b58-4a77-9899-8d3f4bb2ea2a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:35.780977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf19a4-96ad-441e-8142-ab569da4938d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:35.781654 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf19a4-96ad-441e-8142-ab569da4938d admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:35.781931 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf19a4-96ad-441e-8142-ab569da4938d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:35.782203 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaf19a4-96ad-441e-8142-ab569da4938d admin admin] PATH_INFO: `/v3/projects/b7254d89b2394949a10412eedd881831/users/b7abc8a6640741e89b47f8ccdc8dfe60/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:35.797154 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf19a4-96ad-441e-8142-ab569da4938d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7254d89b2394949a10412eedd881831, user_id=b7abc8a6640741e89b47f8ccdc8dfe60, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:35.799804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaf19a4-96ad-441e-8142-ab569da4938d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:35.812786 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1392/2772] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:35 2022] PUT /identity/v3/projects/b7254d89b2394949a10412eedd881831/users/b7abc8a6640741e89b47f8ccdc8dfe60/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:35.822868 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c4d80-65b8-4107-90bf-8f5e366d1ad2 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:35.823360 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c4d80-65b8-4107-90bf-8f5e366d1ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:35.823459 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708c4d80-65b8-4107-90bf-8f5e366d1ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:35.851514 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-708c4d80-65b8-4107-90bf-8f5e366d1ad2 None None] MFA Rules not processed for user `b7abc8a6640741e89b47f8ccdc8dfe60`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:35.852833 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-708c4d80-65b8-4107-90bf-8f5e366d1ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:35.918702 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1381/2773] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:35.923217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf19a4-96ad-441e-8142-ab569da4938d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:35.983239 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaf19a4-96ad-441e-8142-ab569da4938d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:35.988202 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce0bf7-5f39-4086-b011-ca529a4276b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:35.988799 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce0bf7-5f39-4086-b011-ca529a4276b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:35.989079 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce0bf7-5f39-4086-b011-ca529a4276b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:35.989353 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bce0bf7-5f39-4086-b011-ca529a4276b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:35.990032 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce0bf7-5f39-4086-b011-ca529a4276b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:35.991926 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bce0bf7-5f39-4086-b011-ca529a4276b2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:36.017622 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1393/2774] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:35 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:36.022434 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c4d80-65b8-4107-90bf-8f5e366d1ad2 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:36.033148 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708c4d80-65b8-4107-90bf-8f5e366d1ad2 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:36.089411 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e111829-68f0-4897-8b36-46ddbdac9df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:36.090054 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e111829-68f0-4897-8b36-46ddbdac9df9 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:36.090334 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e111829-68f0-4897-8b36-46ddbdac9df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:36.090607 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e111829-68f0-4897-8b36-46ddbdac9df9 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:36.091270 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e111829-68f0-4897-8b36-46ddbdac9df9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:36.093171 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e111829-68f0-4897-8b36-46ddbdac9df9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:36.103420 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:36.115644 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1382/2775] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:36 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:36.119729 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce0bf7-5f39-4086-b011-ca529a4276b2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:36.130146 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bce0bf7-5f39-4086-b011-ca529a4276b2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:36.134676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e7ae9-4f07-4d13-9ae4-a114e112e369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:36.135273 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e7ae9-4f07-4d13-9ae4-a114e112e369 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:36.135551 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e7ae9-4f07-4d13-9ae4-a114e112e369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:36.135824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e7ae9-4f07-4d13-9ae4-a114e112e369 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:36.136542 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e7ae9-4f07-4d13-9ae4-a114e112e369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:36.138175 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e7ae9-4f07-4d13-9ae4-a114e112e369 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:36.149145 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1394/2776] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:36.153194 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e111829-68f0-4897-8b36-46ddbdac9df9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:36.164227 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e111829-68f0-4897-8b36-46ddbdac9df9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:36.168984 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660f90f7-d558-4421-ab3e-809964455df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:36.169628 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f90f7-d558-4421-ab3e-809964455df5 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:36.169914 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f90f7-d558-4421-ab3e-809964455df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:36.170190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660f90f7-d558-4421-ab3e-809964455df5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:36.170882 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f90f7-d558-4421-ab3e-809964455df5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:36.172476 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660f90f7-d558-4421-ab3e-809964455df5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:36.182674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1383/2777] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:36.186700 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e7ae9-4f07-4d13-9ae4-a114e112e369 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:36.196816 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e7ae9-4f07-4d13-9ae4-a114e112e369 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:36.201280 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbe4e4-0717-4535-bcce-91874432c041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:36.201963 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbe4e4-0717-4535-bcce-91874432c041 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:36.202243 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbe4e4-0717-4535-bcce-91874432c041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:36.202513 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbe4e4-0717-4535-bcce-91874432c041 admin admin] PATH_INFO: `/v3/projects/97a1f7f74c8f4999a5867649d933c15f/users/ccac0f7c03a54a01b5893137785f6831/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:36.217927 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afbe4e4-0717-4535-bcce-91874432c041 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97a1f7f74c8f4999a5867649d933c15f, user_id=ccac0f7c03a54a01b5893137785f6831, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:36.221011 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afbe4e4-0717-4535-bcce-91874432c041 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:36.234335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1395/2778] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:36 2022] PUT /identity/v3/projects/97a1f7f74c8f4999a5867649d933c15f/users/ccac0f7c03a54a01b5893137785f6831/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:36.244300 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0f580-8f88-4228-a4e2-705b574274dd None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:36.244585 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0f580-8f88-4228-a4e2-705b574274dd None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:36.244860 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0f580-8f88-4228-a4e2-705b574274dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:36.277648 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-0cd0f580-8f88-4228-a4e2-705b574274dd None None] MFA Rules not processed for user `ccac0f7c03a54a01b5893137785f6831`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:36.278988 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0cd0f580-8f88-4228-a4e2-705b574274dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:36.345177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1384/2779] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:36:36.350377 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbe4e4-0717-4535-bcce-91874432c041 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:36.393227 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:36.420869 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbe4e4-0717-4535-bcce-91874432c041 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:36.427029 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535785a-4455-4c2b-95f3-948fa9a2eb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:36.427672 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535785a-4455-4c2b-95f3-948fa9a2eb35 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:36.428066 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535785a-4455-4c2b-95f3-948fa9a2eb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:36.428203 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535785a-4455-4c2b-95f3-948fa9a2eb35 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:36.428946 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535785a-4455-4c2b-95f3-948fa9a2eb35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:36.430873 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535785a-4455-4c2b-95f3-948fa9a2eb35 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:36.459490 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1396/2780] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:36 2022] POST /identity/v3/projects => generated 377 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:36.467594 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0f580-8f88-4228-a4e2-705b574274dd None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:36.479300 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0f580-8f88-4228-a4e2-705b574274dd None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:36.537314 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc095f06-5629-4698-8040-3593ad5d9803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:36.537959 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc095f06-5629-4698-8040-3593ad5d9803 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:36.538245 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc095f06-5629-4698-8040-3593ad5d9803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:36.538520 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc095f06-5629-4698-8040-3593ad5d9803 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:36.539183 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc095f06-5629-4698-8040-3593ad5d9803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:36.541091 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc095f06-5629-4698-8040-3593ad5d9803 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:36.562228 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1385/2781] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:36 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:36.566973 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535785a-4455-4c2b-95f3-948fa9a2eb35 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:36.577270 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535785a-4455-4c2b-95f3-948fa9a2eb35 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:36.582860 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51852d-2985-41ed-a261-38f9e000bab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:36.583462 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51852d-2985-41ed-a261-38f9e000bab9 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:36.583763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51852d-2985-41ed-a261-38f9e000bab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:36.584040 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f51852d-2985-41ed-a261-38f9e000bab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:36.584733 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51852d-2985-41ed-a261-38f9e000bab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:36.586351 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f51852d-2985-41ed-a261-38f9e000bab9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:36.597753 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1397/2782] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:36.601744 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc095f06-5629-4698-8040-3593ad5d9803 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:36.611735 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc095f06-5629-4698-8040-3593ad5d9803 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:36.616457 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88f55d-70a5-4b5b-ab55-20c9717f2fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:36.617117 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88f55d-70a5-4b5b-ab55-20c9717f2fcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:36.617400 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88f55d-70a5-4b5b-ab55-20c9717f2fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:36.617705 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b88f55d-70a5-4b5b-ab55-20c9717f2fcf admin admin] PATH_INFO: `/v3/projects/b307008b849c4332a75d292ffb055cd2/users/888f52eb7e6945b18313ff186cc63c95/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:36.633807 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88f55d-70a5-4b5b-ab55-20c9717f2fcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b307008b849c4332a75d292ffb055cd2, user_id=888f52eb7e6945b18313ff186cc63c95, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:36.636403 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b88f55d-70a5-4b5b-ab55-20c9717f2fcf admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:36.649853 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1386/2783] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:36 2022] PUT /identity/v3/projects/b307008b849c4332a75d292ffb055cd2/users/888f52eb7e6945b18313ff186cc63c95/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:36.654428 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51852d-2985-41ed-a261-38f9e000bab9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:36.664548 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f51852d-2985-41ed-a261-38f9e000bab9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:36.719540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ba8c35-8cd1-46ab-b0c8-51aef32f3138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:36.720145 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba8c35-8cd1-46ab-b0c8-51aef32f3138 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:36.720423 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba8c35-8cd1-46ab-b0c8-51aef32f3138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:36.720692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ba8c35-8cd1-46ab-b0c8-51aef32f3138 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:36.721382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba8c35-8cd1-46ab-b0c8-51aef32f3138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:36.723676 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ba8c35-8cd1-46ab-b0c8-51aef32f3138 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:36.733140 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1398/2784] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:36:36.737266 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88f55d-70a5-4b5b-ab55-20c9717f2fcf admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:36.748094 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b88f55d-70a5-4b5b-ab55-20c9717f2fcf admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:36.753001 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724358d0-c814-4839-9c02-c0356acc2c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:36.754289 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724358d0-c814-4839-9c02-c0356acc2c97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:36.754565 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724358d0-c814-4839-9c02-c0356acc2c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:36.754834 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724358d0-c814-4839-9c02-c0356acc2c97 admin admin] PATH_INFO: `/v3/projects/b307008b849c4332a75d292ffb055cd2/users/888f52eb7e6945b18313ff186cc63c95/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:36.756792 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724358d0-c814-4839-9c02-c0356acc2c97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b307008b849c4332a75d292ffb055cd2, user_id=888f52eb7e6945b18313ff186cc63c95, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:36.759569 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724358d0-c814-4839-9c02-c0356acc2c97 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:36.769569 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1387/2785] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:36 2022] PUT /identity/v3/projects/b307008b849c4332a75d292ffb055cd2/users/888f52eb7e6945b18313ff186cc63c95/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:36.780256 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d392a-54ff-48e4-94b6-4cf979373035 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:36.780540 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d392a-54ff-48e4-94b6-4cf979373035 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:36.780814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d392a-54ff-48e4-94b6-4cf979373035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:36.807904 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5a6d392a-54ff-48e4-94b6-4cf979373035 None None] MFA Rules not processed for user `888f52eb7e6945b18313ff186cc63c95`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:36.809145 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5a6d392a-54ff-48e4-94b6-4cf979373035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:36.875505 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1399/2786] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:36:36.887533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724358d0-c814-4839-9c02-c0356acc2c97 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:36.889177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-724358d0-c814-4839-9c02-c0356acc2c97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:36.917469 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:36.948415 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724358d0-c814-4839-9c02-c0356acc2c97 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:36.952265 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191804d-65df-444c-bac0-b168bdafaf52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:36.952874 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191804d-65df-444c-bac0-b168bdafaf52 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:36.953152 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191804d-65df-444c-bac0-b168bdafaf52 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:36.953470 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191804d-65df-444c-bac0-b168bdafaf52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:37.010208 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5191804d-65df-444c-bac0-b168bdafaf52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:37.011712 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5191804d-65df-444c-bac0-b168bdafaf52 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:37.021581 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1388/2787] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:38.611804 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d392a-54ff-48e4-94b6-4cf979373035 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:38.614011 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5a6d392a-54ff-48e4-94b6-4cf979373035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:38.629682 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d392a-54ff-48e4-94b6-4cf979373035 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:38.633312 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b287c3a-384f-4ec1-ada1-aaf419472f5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:38.633960 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b287c3a-384f-4ec1-ada1-aaf419472f5f service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:38.634244 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b287c3a-384f-4ec1-ada1-aaf419472f5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:38.634512 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b287c3a-384f-4ec1-ada1-aaf419472f5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:38.668483 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:38.709288 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b287c3a-384f-4ec1-ada1-aaf419472f5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:38.709288 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b287c3a-384f-4ec1-ada1-aaf419472f5f service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:38.715292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1400/2788] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:39.275416 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191804d-65df-444c-bac0-b168bdafaf52 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:39.284292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191804d-65df-444c-bac0-b168bdafaf52 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:39.288899 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e830b-0f32-49fa-b55e-feb2fc14be81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:39.289509 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e830b-0f32-49fa-b55e-feb2fc14be81 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:39.289822 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e830b-0f32-49fa-b55e-feb2fc14be81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:39.290098 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e830b-0f32-49fa-b55e-feb2fc14be81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:39.299151 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e830b-0f32-49fa-b55e-feb2fc14be81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:39.301674 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e830b-0f32-49fa-b55e-feb2fc14be81 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:39.311122 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1389/2789] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:39 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:47.431561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b287c3a-384f-4ec1-ada1-aaf419472f5f service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:47.496692 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b287c3a-384f-4ec1-ada1-aaf419472f5f service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:47.501181 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff239a-7644-4c9b-aa5a-c351bca38126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:47.501852 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff239a-7644-4c9b-aa5a-c351bca38126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:47.502136 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff239a-7644-4c9b-aa5a-c351bca38126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:47.502410 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ff239a-7644-4c9b-aa5a-c351bca38126 admin admin] PATH_INFO: `/v3/users/b7abc8a6640741e89b47f8ccdc8dfe60` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:47.503815 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff239a-7644-4c9b-aa5a-c351bca38126 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7abc8a6640741e89b47f8ccdc8dfe60)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:47.506902 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ff239a-7644-4c9b-aa5a-c351bca38126 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:47.553223 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d5ff239a-7644-4c9b-aa5a-c351bca38126 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7abc8a6640741e89b47f8ccdc8dfe60'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:47.560005 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d5ff239a-7644-4c9b-aa5a-c351bca38126 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7abc8a6640741e89b47f8ccdc8dfe60'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:47.560655 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d5ff239a-7644-4c9b-aa5a-c351bca38126 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7abc8a6640741e89b47f8ccdc8dfe60'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:47.561258 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d5ff239a-7644-4c9b-aa5a-c351bca38126 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7abc8a6640741e89b47f8ccdc8dfe60'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:47.573476 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d5ff239a-7644-4c9b-aa5a-c351bca38126 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7abc8a6640741e89b47f8ccdc8dfe60'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:47.586174 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1401/2790] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:47 2022] DELETE /identity/v3/users/b7abc8a6640741e89b47f8ccdc8dfe60 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:47.590904 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2d768-6d7c-4104-bcc3-8582f805bc9a None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:47.591384 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2d768-6d7c-4104-bcc3-8582f805bc9a None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:47.591487 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e2d768-6d7c-4104-bcc3-8582f805bc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:47.610681 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-18e2d768-6d7c-4104-bcc3-8582f805bc9a None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:47.611870 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-18e2d768-6d7c-4104-bcc3-8582f805bc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:47.669219 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1390/2791] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:47.679784 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff239a-7644-4c9b-aa5a-c351bca38126 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:47.681184 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d5ff239a-7644-4c9b-aa5a-c351bca38126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:47.732923 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:47.737626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ff239a-7644-4c9b-aa5a-c351bca38126 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:47.743368 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688c90a-5404-41f0-82eb-82a3948a7f05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:47.744125 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688c90a-5404-41f0-82eb-82a3948a7f05 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:47.745171 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688c90a-5404-41f0-82eb-82a3948a7f05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:47.745421 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9688c90a-5404-41f0-82eb-82a3948a7f05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:47.758703 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688c90a-5404-41f0-82eb-82a3948a7f05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:47.760220 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9688c90a-5404-41f0-82eb-82a3948a7f05 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:47.771456 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1402/2792] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:48.037570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2d768-6d7c-4104-bcc3-8582f805bc9a None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:48.039009 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-18e2d768-6d7c-4104-bcc3-8582f805bc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:48.052410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e2d768-6d7c-4104-bcc3-8582f805bc9a None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:48.057186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f386477-f6f4-4979-b1cb-b1a9bb46ac98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:48.057974 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f386477-f6f4-4979-b1cb-b1a9bb46ac98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:48.058263 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f386477-f6f4-4979-b1cb-b1a9bb46ac98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:48.058540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f386477-f6f4-4979-b1cb-b1a9bb46ac98 admin admin] PATH_INFO: `/v3/projects/b7254d89b2394949a10412eedd881831` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:48.060913 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f386477-f6f4-4979-b1cb-b1a9bb46ac98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7254d89b2394949a10412eedd881831)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:48.062907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f386477-f6f4-4979-b1cb-b1a9bb46ac98 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:48.111352 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3f386477-f6f4-4979-b1cb-b1a9bb46ac98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7254d89b2394949a10412eedd881831', 'request_id': 'req-3f386477-f6f4-4979-b1cb-b1a9bb46ac98'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:48.119109 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3f386477-f6f4-4979-b1cb-b1a9bb46ac98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7254d89b2394949a10412eedd881831', 'request_id': 'req-3f386477-f6f4-4979-b1cb-b1a9bb46ac98'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:48.123158 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:48.125514 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3f386477-f6f4-4979-b1cb-b1a9bb46ac98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7254d89b2394949a10412eedd881831', 'request_id': 'req-3f386477-f6f4-4979-b1cb-b1a9bb46ac98'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:48.138077 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3f386477-f6f4-4979-b1cb-b1a9bb46ac98 admin admin] The token cache is being invalidate because project b7254d89b2394949a10412eedd881831 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:48.138370 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3f386477-f6f4-4979-b1cb-b1a9bb46ac98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:48.139006 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3f386477-f6f4-4979-b1cb-b1a9bb46ac98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:48.140614 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1391/2793] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:48 2022] DELETE /identity/v3/projects/b7254d89b2394949a10412eedd881831 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:48.144564 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688c90a-5404-41f0-82eb-82a3948a7f05 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:48.154800 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9688c90a-5404-41f0-82eb-82a3948a7f05 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:48.211051 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef8798-40a8-4af9-acc3-0c6adf67799c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:48.211609 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef8798-40a8-4af9-acc3-0c6adf67799c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:48.211885 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef8798-40a8-4af9-acc3-0c6adf67799c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:48.212147 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef8798-40a8-4af9-acc3-0c6adf67799c admin admin] PATH_INFO: `/v3/users/ccac0f7c03a54a01b5893137785f6831` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:48.213291 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef8798-40a8-4af9-acc3-0c6adf67799c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccac0f7c03a54a01b5893137785f6831)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:48.215292 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef8798-40a8-4af9-acc3-0c6adf67799c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:48.260543 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d0ef8798-40a8-4af9-acc3-0c6adf67799c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccac0f7c03a54a01b5893137785f6831'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:48.267488 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d0ef8798-40a8-4af9-acc3-0c6adf67799c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccac0f7c03a54a01b5893137785f6831'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:48.268164 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d0ef8798-40a8-4af9-acc3-0c6adf67799c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccac0f7c03a54a01b5893137785f6831'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:48.268768 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d0ef8798-40a8-4af9-acc3-0c6adf67799c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccac0f7c03a54a01b5893137785f6831'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:48.280559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d0ef8798-40a8-4af9-acc3-0c6adf67799c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccac0f7c03a54a01b5893137785f6831'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:48.295203 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1403/2794] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:48 2022] DELETE /identity/v3/users/ccac0f7c03a54a01b5893137785f6831 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:48.563050 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f386477-f6f4-4979-b1cb-b1a9bb46ac98 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:48.564522 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3f386477-f6f4-4979-b1cb-b1a9bb46ac98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:48.580606 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f386477-f6f4-4979-b1cb-b1a9bb46ac98 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:48.641883 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f2347-a611-4329-b52a-edf1a1e13927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:48.642589 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f2347-a611-4329-b52a-edf1a1e13927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:48.642797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f2347-a611-4329-b52a-edf1a1e13927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:48.643064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f2347-a611-4329-b52a-edf1a1e13927 admin admin] PATH_INFO: `/v3/projects/97a1f7f74c8f4999a5867649d933c15f` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:48.644222 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f2347-a611-4329-b52a-edf1a1e13927 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97a1f7f74c8f4999a5867649d933c15f)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:48.646237 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5f2347-a611-4329-b52a-edf1a1e13927 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:48.692972 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6b5f2347-a611-4329-b52a-edf1a1e13927 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97a1f7f74c8f4999a5867649d933c15f', 'request_id': 'req-6b5f2347-a611-4329-b52a-edf1a1e13927'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:48.700142 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6b5f2347-a611-4329-b52a-edf1a1e13927 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97a1f7f74c8f4999a5867649d933c15f', 'request_id': 'req-6b5f2347-a611-4329-b52a-edf1a1e13927'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:48.705748 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6b5f2347-a611-4329-b52a-edf1a1e13927 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97a1f7f74c8f4999a5867649d933c15f', 'request_id': 'req-6b5f2347-a611-4329-b52a-edf1a1e13927'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:48.717119 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6b5f2347-a611-4329-b52a-edf1a1e13927 admin admin] The token cache is being invalidate because project 97a1f7f74c8f4999a5867649d933c15f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:48.717405 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6b5f2347-a611-4329-b52a-edf1a1e13927 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:48.718073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-6b5f2347-a611-4329-b52a-edf1a1e13927 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:48.719567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1392/2795] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:48 2022] DELETE /identity/v3/projects/97a1f7f74c8f4999a5867649d933c15f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:48.724187 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef8798-40a8-4af9-acc3-0c6adf67799c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:48.725754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d0ef8798-40a8-4af9-acc3-0c6adf67799c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:48.738444 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef8798-40a8-4af9-acc3-0c6adf67799c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:48.800965 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f29f01-3759-4dd8-a998-664c1fad73f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:48.801828 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f29f01-3759-4dd8-a998-664c1fad73f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:48.802332 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f29f01-3759-4dd8-a998-664c1fad73f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:48.802332 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f29f01-3759-4dd8-a998-664c1fad73f0 admin admin] PATH_INFO: `/v3/users/888f52eb7e6945b18313ff186cc63c95` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:48.803463 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f29f01-3759-4dd8-a998-664c1fad73f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=888f52eb7e6945b18313ff186cc63c95)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:48.805374 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f29f01-3759-4dd8-a998-664c1fad73f0 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:48.817736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:48.855355 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e9f29f01-3759-4dd8-a998-664c1fad73f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '888f52eb7e6945b18313ff186cc63c95'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:48.862194 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e9f29f01-3759-4dd8-a998-664c1fad73f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '888f52eb7e6945b18313ff186cc63c95'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:48.862871 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e9f29f01-3759-4dd8-a998-664c1fad73f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '888f52eb7e6945b18313ff186cc63c95'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:48.863598 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e9f29f01-3759-4dd8-a998-664c1fad73f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '888f52eb7e6945b18313ff186cc63c95'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:48.875853 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e9f29f01-3759-4dd8-a998-664c1fad73f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '888f52eb7e6945b18313ff186cc63c95'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:48.888436 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1404/2796] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:48 2022] DELETE /identity/v3/users/888f52eb7e6945b18313ff186cc63c95 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:49.163225 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f2347-a611-4329-b52a-edf1a1e13927 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:49.164867 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-6b5f2347-a611-4329-b52a-edf1a1e13927 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:49.178493 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f2347-a611-4329-b52a-edf1a1e13927 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:49.235627 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b81cd-589f-48e2-b17d-0def1a36954b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:49.236184 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b81cd-589f-48e2-b17d-0def1a36954b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:49.236464 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b81cd-589f-48e2-b17d-0def1a36954b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:49.236736 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b81cd-589f-48e2-b17d-0def1a36954b admin admin] PATH_INFO: `/v3/projects/b307008b849c4332a75d292ffb055cd2` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:49.237910 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b81cd-589f-48e2-b17d-0def1a36954b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b307008b849c4332a75d292ffb055cd2)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:49.240582 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b81cd-589f-48e2-b17d-0def1a36954b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:49.262506 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:49.287466 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-579b81cd-589f-48e2-b17d-0def1a36954b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b307008b849c4332a75d292ffb055cd2', 'request_id': 'req-579b81cd-589f-48e2-b17d-0def1a36954b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:49.294417 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-579b81cd-589f-48e2-b17d-0def1a36954b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b307008b849c4332a75d292ffb055cd2', 'request_id': 'req-579b81cd-589f-48e2-b17d-0def1a36954b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:49.299867 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-579b81cd-589f-48e2-b17d-0def1a36954b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b307008b849c4332a75d292ffb055cd2', 'request_id': 'req-579b81cd-589f-48e2-b17d-0def1a36954b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:49.310712 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-579b81cd-589f-48e2-b17d-0def1a36954b admin admin] The token cache is being invalidate because project b307008b849c4332a75d292ffb055cd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:49.311000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-579b81cd-589f-48e2-b17d-0def1a36954b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:49.311671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-579b81cd-589f-48e2-b17d-0def1a36954b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:49.313220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1393/2797] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:49 2022] DELETE /identity/v3/projects/b307008b849c4332a75d292ffb055cd2 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:49.318726 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e6d6e-e763-4978-b700-3889adec3444 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:49.319011 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e6d6e-e763-4978-b700-3889adec3444 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:49.319284 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351e6d6e-e763-4978-b700-3889adec3444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:49.338728 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-351e6d6e-e763-4978-b700-3889adec3444 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:49.339913 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-351e6d6e-e763-4978-b700-3889adec3444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:49.399108 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1405/2798] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:49.407136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc37fc1-ada3-41e3-b13c-94d35e4df275 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:49.407421 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc37fc1-ada3-41e3-b13c-94d35e4df275 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:49.407699 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc37fc1-ada3-41e3-b13c-94d35e4df275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:49.427972 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-3bc37fc1-ada3-41e3-b13c-94d35e4df275 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:49.429162 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3bc37fc1-ada3-41e3-b13c-94d35e4df275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:49.437018 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1394/2799] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:49.441340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e6d6e-e763-4978-b700-3889adec3444 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:49.451670 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351e6d6e-e763-4978-b700-3889adec3444 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:49.456251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec105bb-10c0-4c6d-b34a-6fc747173dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:49.456855 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec105bb-10c0-4c6d-b34a-6fc747173dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:49.457138 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec105bb-10c0-4c6d-b34a-6fc747173dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:49.457412 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec105bb-10c0-4c6d-b34a-6fc747173dc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:49.458185 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec105bb-10c0-4c6d-b34a-6fc747173dc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:36:49.458475 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec105bb-10c0-4c6d-b34a-6fc747173dc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:49.460060 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec105bb-10c0-4c6d-b34a-6fc747173dc3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:49.471454 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1406/2800] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:36:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:36:49.475885 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc37fc1-ada3-41e3-b13c-94d35e4df275 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:49.485290 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc37fc1-ada3-41e3-b13c-94d35e4df275 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:49.491209 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730b275-721f-44f8-a686-45a6b3d6383b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:49.491757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730b275-721f-44f8-a686-45a6b3d6383b admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:49.492041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730b275-721f-44f8-a686-45a6b3d6383b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:49.492317 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730b275-721f-44f8-a686-45a6b3d6383b admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:49.492977 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730b275-721f-44f8-a686-45a6b3d6383b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:49.494896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730b275-721f-44f8-a686-45a6b3d6383b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:49.524991 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1395/2801] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:49 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:36:49.529116 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec105bb-10c0-4c6d-b34a-6fc747173dc3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:49.540999 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec105bb-10c0-4c6d-b34a-6fc747173dc3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:49.597639 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801d51f-987d-447e-91c0-72270ea15963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:49.598279 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801d51f-987d-447e-91c0-72270ea15963 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:49.598635 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801d51f-987d-447e-91c0-72270ea15963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:49.598916 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f801d51f-987d-447e-91c0-72270ea15963 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:49.599574 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f801d51f-987d-447e-91c0-72270ea15963 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:49.601565 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f801d51f-987d-447e-91c0-72270ea15963 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:49.624106 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1407/2802] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:49 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:49.629233 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730b275-721f-44f8-a686-45a6b3d6383b admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:49.641071 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730b275-721f-44f8-a686-45a6b3d6383b admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:49.645925 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da57a3-0049-40e5-823d-2c0cd917e1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:49.646614 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da57a3-0049-40e5-823d-2c0cd917e1dc admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:49.646901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da57a3-0049-40e5-823d-2c0cd917e1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:49.647180 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4da57a3-0049-40e5-823d-2c0cd917e1dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:49.647932 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da57a3-0049-40e5-823d-2c0cd917e1dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:49.649567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4da57a3-0049-40e5-823d-2c0cd917e1dc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:49.659576 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1396/2803] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:49.663844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801d51f-987d-447e-91c0-72270ea15963 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:49.674723 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f801d51f-987d-447e-91c0-72270ea15963 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:49.680024 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa009c4-0d22-4f1e-8782-49a050f6cade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:49.680633 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa009c4-0d22-4f1e-8782-49a050f6cade admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:49.680910 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa009c4-0d22-4f1e-8782-49a050f6cade admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:49.681178 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa009c4-0d22-4f1e-8782-49a050f6cade admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:49.681896 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa009c4-0d22-4f1e-8782-49a050f6cade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:49.683469 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa009c4-0d22-4f1e-8782-49a050f6cade admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:49.684739 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:49.696348 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1408/2804] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:49.700358 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da57a3-0049-40e5-823d-2c0cd917e1dc admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:49.710504 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4da57a3-0049-40e5-823d-2c0cd917e1dc admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:49.715131 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabc121-3ce8-4d15-9f75-75c97b897dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:49.715790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabc121-3ce8-4d15-9f75-75c97b897dfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:49.716076 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabc121-3ce8-4d15-9f75-75c97b897dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:49.716354 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabc121-3ce8-4d15-9f75-75c97b897dfe admin admin] PATH_INFO: `/v3/projects/651fa66c961b4d07a8fa735e63cdef79/users/2e64644dc881469e907a6f6669999757/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:49.730918 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabc121-3ce8-4d15-9f75-75c97b897dfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=651fa66c961b4d07a8fa735e63cdef79, user_id=2e64644dc881469e907a6f6669999757, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:49.733639 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edabc121-3ce8-4d15-9f75-75c97b897dfe admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:49.747348 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1397/2805] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:49 2022] PUT /identity/v3/projects/651fa66c961b4d07a8fa735e63cdef79/users/2e64644dc881469e907a6f6669999757/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:49.752280 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10b260-4340-4d8d-b6d7-092bdba1681a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:49.752988 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10b260-4340-4d8d-b6d7-092bdba1681a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:49.753120 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10b260-4340-4d8d-b6d7-092bdba1681a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:49.772247 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-af10b260-4340-4d8d-b6d7-092bdba1681a None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:49.773484 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-af10b260-4340-4d8d-b6d7-092bdba1681a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:49.832618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1409/2806] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:49.842437 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabc121-3ce8-4d15-9f75-75c97b897dfe admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:49.844867 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-edabc121-3ce8-4d15-9f75-75c97b897dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:49.903559 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabc121-3ce8-4d15-9f75-75c97b897dfe admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:49.907540 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621f0d5-e51e-4e79-a70b-1b953d7ee7d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:49.908192 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621f0d5-e51e-4e79-a70b-1b953d7ee7d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:49.908473 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621f0d5-e51e-4e79-a70b-1b953d7ee7d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:49.908743 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621f0d5-e51e-4e79-a70b-1b953d7ee7d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:49.919519 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621f0d5-e51e-4e79-a70b-1b953d7ee7d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:49.920954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7621f0d5-e51e-4e79-a70b-1b953d7ee7d9 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:49.931111 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1398/2807] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:50.599023 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10b260-4340-4d8d-b6d7-092bdba1681a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:50.601234 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-af10b260-4340-4d8d-b6d7-092bdba1681a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:50.616007 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10b260-4340-4d8d-b6d7-092bdba1681a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:50.621063 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13826528-5b1d-48a0-ae61-f2ebb70c64d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:50.622884 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13826528-5b1d-48a0-ae61-f2ebb70c64d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:50.623170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13826528-5b1d-48a0-ae61-f2ebb70c64d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:50.623445 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13826528-5b1d-48a0-ae61-f2ebb70c64d9 admin admin] PATH_INFO: `/v3/users/b32e87e5d8234756bfb63e60854331d2` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:50.624966 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13826528-5b1d-48a0-ae61-f2ebb70c64d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b32e87e5d8234756bfb63e60854331d2)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:50.628726 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13826528-5b1d-48a0-ae61-f2ebb70c64d9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:50.713644 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-13826528-5b1d-48a0-ae61-f2ebb70c64d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b32e87e5d8234756bfb63e60854331d2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:50.724983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-13826528-5b1d-48a0-ae61-f2ebb70c64d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b32e87e5d8234756bfb63e60854331d2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:50.726268 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-13826528-5b1d-48a0-ae61-f2ebb70c64d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b32e87e5d8234756bfb63e60854331d2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:50.727162 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-13826528-5b1d-48a0-ae61-f2ebb70c64d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b32e87e5d8234756bfb63e60854331d2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:50.746977 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-13826528-5b1d-48a0-ae61-f2ebb70c64d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b32e87e5d8234756bfb63e60854331d2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:50.775952 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1410/2808] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:36:50 2022] DELETE /identity/v3/users/b32e87e5d8234756bfb63e60854331d2 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:50.780815 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac50c2-eb71-41ce-b7af-118afae58c38 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:50.781296 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac50c2-eb71-41ce-b7af-118afae58c38 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:50.781413 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac50c2-eb71-41ce-b7af-118afae58c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:50.801186 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-f1ac50c2-eb71-41ce-b7af-118afae58c38 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:50.802925 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f1ac50c2-eb71-41ce-b7af-118afae58c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:50.854624 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:50.881937 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1399/2809] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:36:50.894505 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13826528-5b1d-48a0-ae61-f2ebb70c64d9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:50.898097 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-13826528-5b1d-48a0-ae61-f2ebb70c64d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:51.010744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13826528-5b1d-48a0-ae61-f2ebb70c64d9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:51.015885 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09e39d-1fae-4be4-bc31-72a710e3b837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:51.016872 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09e39d-1fae-4be4-bc31-72a710e3b837 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:51.017261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09e39d-1fae-4be4-bc31-72a710e3b837 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:51.017720 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09e39d-1fae-4be4-bc31-72a710e3b837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:51.042041 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09e39d-1fae-4be4-bc31-72a710e3b837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:51.045140 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09e39d-1fae-4be4-bc31-72a710e3b837 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:51.060695 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1411/2810] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:51.384836 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac50c2-eb71-41ce-b7af-118afae58c38 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:51.387200 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-f1ac50c2-eb71-41ce-b7af-118afae58c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:51.406559 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac50c2-eb71-41ce-b7af-118afae58c38 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:51.412755 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5804b1c9-4922-4392-8007-30210bf0184b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:51.413559 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804b1c9-4922-4392-8007-30210bf0184b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:51.413986 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804b1c9-4922-4392-8007-30210bf0184b admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:51.415018 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5804b1c9-4922-4392-8007-30210bf0184b admin admin] PATH_INFO: `/v3/projects/39ffbd016b784495af3b1be258c5ff6f` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:51.415915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804b1c9-4922-4392-8007-30210bf0184b admin admin] RBAC: Authorizing `identity:delete_project(project_id=39ffbd016b784495af3b1be258c5ff6f)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:51.418541 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5804b1c9-4922-4392-8007-30210bf0184b admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:51.482710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5804b1c9-4922-4392-8007-30210bf0184b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39ffbd016b784495af3b1be258c5ff6f', 'request_id': 'req-5804b1c9-4922-4392-8007-30210bf0184b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:51.495201 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5804b1c9-4922-4392-8007-30210bf0184b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39ffbd016b784495af3b1be258c5ff6f', 'request_id': 'req-5804b1c9-4922-4392-8007-30210bf0184b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:51.510996 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5804b1c9-4922-4392-8007-30210bf0184b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39ffbd016b784495af3b1be258c5ff6f', 'request_id': 'req-5804b1c9-4922-4392-8007-30210bf0184b'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:51.525567 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5804b1c9-4922-4392-8007-30210bf0184b admin admin] The token cache is being invalidate because project 39ffbd016b784495af3b1be258c5ff6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:36:51.525858 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5804b1c9-4922-4392-8007-30210bf0184b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:51.526563 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-5804b1c9-4922-4392-8007-30210bf0184b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:36:51.531759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1400/2811] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:36:51 2022] DELETE /identity/v3/projects/39ffbd016b784495af3b1be258c5ff6f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:51.533621 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5bf25-5309-402e-a158-0e624d6ba503 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:51.534027 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5bf25-5309-402e-a158-0e624d6ba503 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:51.534384 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5bf25-5309-402e-a158-0e624d6ba503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:51.558394 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8cb5bf25-5309-402e-a158-0e624d6ba503 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:51.559695 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8cb5bf25-5309-402e-a158-0e624d6ba503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:51.565666 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:51.638859 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1412/2812] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:51.647667 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ef356-c073-4ccd-b0c8-2ac76938f7eb None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:51.648953 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ef356-c073-4ccd-b0c8-2ac76938f7eb None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:51.648953 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ef356-c073-4ccd-b0c8-2ac76938f7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:51.700097 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-750ef356-c073-4ccd-b0c8-2ac76938f7eb None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:51.701305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-750ef356-c073-4ccd-b0c8-2ac76938f7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:51.710088 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1401/2813] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:51.714777 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5bf25-5309-402e-a158-0e624d6ba503 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:51.730313 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5bf25-5309-402e-a158-0e624d6ba503 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:51.738776 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be50230-faee-4423-9a33-b36dee805502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:51.740161 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be50230-faee-4423-9a33-b36dee805502 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:51.740452 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be50230-faee-4423-9a33-b36dee805502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:51.740730 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be50230-faee-4423-9a33-b36dee805502 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:51.742104 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be50230-faee-4423-9a33-b36dee805502 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:36:51.742104 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be50230-faee-4423-9a33-b36dee805502 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:51.743546 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be50230-faee-4423-9a33-b36dee805502 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:51.764543 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1413/2814] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 28 22:36:51.769569 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ef356-c073-4ccd-b0c8-2ac76938f7eb None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:51.779609 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ef356-c073-4ccd-b0c8-2ac76938f7eb None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:51.785219 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0400f4-f0a4-4b90-ab5e-81bb4961ffb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:51.786659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0400f4-f0a4-4b90-ab5e-81bb4961ffb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:51.787410 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0400f4-f0a4-4b90-ab5e-81bb4961ffb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:51.787892 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0400f4-f0a4-4b90-ab5e-81bb4961ffb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:51.788622 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0400f4-f0a4-4b90-ab5e-81bb4961ffb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:51.790931 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0400f4-f0a4-4b90-ab5e-81bb4961ffb5 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:51.833382 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1402/2815] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:51 2022] POST /identity/v3/projects => generated 389 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:51.834853 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be50230-faee-4423-9a33-b36dee805502 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:51.846638 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be50230-faee-4423-9a33-b36dee805502 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:51.913745 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e5775-2d1f-464d-8b4e-2f272564e65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:51.914405 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e5775-2d1f-464d-8b4e-2f272564e65e admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:51.914779 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e5775-2d1f-464d-8b4e-2f272564e65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:51.915188 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839e5775-2d1f-464d-8b4e-2f272564e65e admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:51.916248 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e5775-2d1f-464d-8b4e-2f272564e65e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:51.918516 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839e5775-2d1f-464d-8b4e-2f272564e65e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:51.942860 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1414/2816] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:51 2022] POST /identity/v3/users => generated 295 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:36:51.947126 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0400f4-f0a4-4b90-ab5e-81bb4961ffb5 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:51.958096 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0400f4-f0a4-4b90-ab5e-81bb4961ffb5 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:51.965110 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41438b34-a507-402e-a222-4baa8651000e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:51.965773 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41438b34-a507-402e-a222-4baa8651000e admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:51.966060 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41438b34-a507-402e-a222-4baa8651000e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:51.966337 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41438b34-a507-402e-a222-4baa8651000e admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:51.967282 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41438b34-a507-402e-a222-4baa8651000e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:51.968796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41438b34-a507-402e-a222-4baa8651000e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:51.984504 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1403/2817] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:51.985206 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e5775-2d1f-464d-8b4e-2f272564e65e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:52.004300 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839e5775-2d1f-464d-8b4e-2f272564e65e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:52.027217 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee3ddb8-15bc-4031-80bb-9ce250000764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:52.028072 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3ddb8-15bc-4031-80bb-9ce250000764 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:52.028426 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3ddb8-15bc-4031-80bb-9ce250000764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:52.028770 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee3ddb8-15bc-4031-80bb-9ce250000764 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:52.029587 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3ddb8-15bc-4031-80bb-9ce250000764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:52.031359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee3ddb8-15bc-4031-80bb-9ce250000764 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:52.048446 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41438b34-a507-402e-a222-4baa8651000e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:52.049946 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1415/2818] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:52.060677 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41438b34-a507-402e-a222-4baa8651000e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:52.071671 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9b3b9-3a72-4b6a-985c-c72e726c63c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:52.072458 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9b3b9-3a72-4b6a-985c-c72e726c63c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:52.072819 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9b3b9-3a72-4b6a-985c-c72e726c63c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:52.073312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9b3b9-3a72-4b6a-985c-c72e726c63c4 admin admin] PATH_INFO: `/v3/projects/c5b2013a802a4661952c1f1260d29e55/users/f44c045f18c04a29a811a313b65ea710/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:52.089782 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9b3b9-3a72-4b6a-985c-c72e726c63c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5b2013a802a4661952c1f1260d29e55, user_id=f44c045f18c04a29a811a313b65ea710, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:52.092615 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9b3b9-3a72-4b6a-985c-c72e726c63c4 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:52.124335 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4e62b-2339-4d35-93d7-1e958abbfa22 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:52.124866 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4e62b-2339-4d35-93d7-1e958abbfa22 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:52.125261 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4e62b-2339-4d35-93d7-1e958abbfa22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:52.129053 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1404/2819] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:52 2022] PUT /identity/v3/projects/c5b2013a802a4661952c1f1260d29e55/users/f44c045f18c04a29a811a313b65ea710/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:36:52.163834 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8be4e62b-2339-4d35-93d7-1e958abbfa22 None None] MFA Rules not processed for user `f44c045f18c04a29a811a313b65ea710`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:52.165089 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8be4e62b-2339-4d35-93d7-1e958abbfa22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:52.323620 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1416/2820] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:52.335692 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9b3b9-3a72-4b6a-985c-c72e726c63c4 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:52.338142 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d5c9b3b9-3a72-4b6a-985c-c72e726c63c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:52.458461 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9b3b9-3a72-4b6a-985c-c72e726c63c4 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:52.468152 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944dfdea-9790-4e4e-9ba0-61d6eaa4cc88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:52.469253 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944dfdea-9790-4e4e-9ba0-61d6eaa4cc88 service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:52.469253 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944dfdea-9790-4e4e-9ba0-61d6eaa4cc88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:52.469553 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944dfdea-9790-4e4e-9ba0-61d6eaa4cc88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:52.480104 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944dfdea-9790-4e4e-9ba0-61d6eaa4cc88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:52.481651 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944dfdea-9790-4e4e-9ba0-61d6eaa4cc88 service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:52.511598 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1405/2821] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:36:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:36:54.525416 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50184fe4-61f1-4872-9d21-84848c2f3a00 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:54.525416 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50184fe4-61f1-4872-9d21-84848c2f3a00 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:54.525983 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50184fe4-61f1-4872-9d21-84848c2f3a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:54.540325 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:54.559965 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-50184fe4-61f1-4872-9d21-84848c2f3a00 None None] MFA Rules not processed for user `2e64644dc881469e907a6f6669999757`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:36:54.561647 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-50184fe4-61f1-4872-9d21-84848c2f3a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:54.651468 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1417/2822] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:36:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:36:54.655993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944dfdea-9790-4e4e-9ba0-61d6eaa4cc88 service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:54.658133 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-944dfdea-9790-4e4e-9ba0-61d6eaa4cc88 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:36:54.673900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:36:54.732652 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944dfdea-9790-4e4e-9ba0-61d6eaa4cc88 service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:54.737424 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0a937-572c-4e2d-8663-3003baaf7dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:54.738158 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0a937-572c-4e2d-8663-3003baaf7dad admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:54.738447 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0a937-572c-4e2d-8663-3003baaf7dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:54.738941 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0a937-572c-4e2d-8663-3003baaf7dad admin admin] PATH_INFO: `/v3/projects` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:54.739757 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c0a937-572c-4e2d-8663-3003baaf7dad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:54.741728 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c0a937-572c-4e2d-8663-3003baaf7dad admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:54.771000 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1406/2823] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:36:54 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:54.775756 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50184fe4-61f1-4872-9d21-84848c2f3a00 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:54.788097 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50184fe4-61f1-4872-9d21-84848c2f3a00 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:54.848065 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f17ed-9116-4ac2-a796-2cacc425372e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:54.849432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f17ed-9116-4ac2-a796-2cacc425372e admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:54.849747 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f17ed-9116-4ac2-a796-2cacc425372e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:54.850026 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f17ed-9116-4ac2-a796-2cacc425372e admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:54.850716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f17ed-9116-4ac2-a796-2cacc425372e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:54.852669 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f17ed-9116-4ac2-a796-2cacc425372e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:54.877282 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1418/2824] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:36:54 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:36:54.882089 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0a937-572c-4e2d-8663-3003baaf7dad admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:54.892983 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0a937-572c-4e2d-8663-3003baaf7dad admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:54.898964 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8d629-a852-4e02-be00-0ea2efdae9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:54.899699 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8d629-a852-4e02-be00-0ea2efdae9f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:54.899983 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8d629-a852-4e02-be00-0ea2efdae9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:54.900257 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af8d629-a852-4e02-be00-0ea2efdae9f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:54.900967 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af8d629-a852-4e02-be00-0ea2efdae9f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:54.902580 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af8d629-a852-4e02-be00-0ea2efdae9f1 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:54.914689 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1407/2825] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:36:54.920128 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f17ed-9116-4ac2-a796-2cacc425372e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:54.930655 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f17ed-9116-4ac2-a796-2cacc425372e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:54.935606 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84de4b-a0f5-42e8-9f37-20fabb7be0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:54.936377 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84de4b-a0f5-42e8-9f37-20fabb7be0ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:54.936736 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84de4b-a0f5-42e8-9f37-20fabb7be0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:54.937121 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84de4b-a0f5-42e8-9f37-20fabb7be0ab admin admin] PATH_INFO: `/v3/projects/f26452741ee2411aa990bc713efc2886/users/51c3e51644fc46ab9befd6eee1accda1/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:54.953943 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84de4b-a0f5-42e8-9f37-20fabb7be0ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f26452741ee2411aa990bc713efc2886, user_id=51c3e51644fc46ab9befd6eee1accda1, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:54.962534 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84de4b-a0f5-42e8-9f37-20fabb7be0ab admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:54.983444 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1419/2826] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:54 2022] PUT /identity/v3/projects/f26452741ee2411aa990bc713efc2886/users/51c3e51644fc46ab9befd6eee1accda1/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:36:54.987516 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8d629-a852-4e02-be00-0ea2efdae9f1 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:54.998526 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af8d629-a852-4e02-be00-0ea2efdae9f1 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:55.070682 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd3684-a962-4660-b88d-382837239caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:55.071506 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd3684-a962-4660-b88d-382837239caf admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:55.071924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd3684-a962-4660-b88d-382837239caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:55.072323 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd3684-a962-4660-b88d-382837239caf admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:55.073387 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd3684-a962-4660-b88d-382837239caf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:55.075898 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd3684-a962-4660-b88d-382837239caf admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:55.090786 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1408/2827] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:36:54 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:36:55.095054 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84de4b-a0f5-42e8-9f37-20fabb7be0ab admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:36:55.108348 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84de4b-a0f5-42e8-9f37-20fabb7be0ab admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:36:55.113097 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67fca3b-444a-4d8b-86ab-12b4c57fa2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:36:55.113727 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fca3b-444a-4d8b-86ab-12b4c57fa2c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:36:55.114119 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fca3b-444a-4d8b-86ab-12b4c57fa2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:36:55.114400 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67fca3b-444a-4d8b-86ab-12b4c57fa2c1 admin admin] PATH_INFO: `/v3/projects/f26452741ee2411aa990bc713efc2886/users/51c3e51644fc46ab9befd6eee1accda1/roles/aded7bd43e974686b3c23bd9d4255b24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:36:55.116579 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67fca3b-444a-4d8b-86ab-12b4c57fa2c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f26452741ee2411aa990bc713efc2886, user_id=51c3e51644fc46ab9befd6eee1accda1, role_id=aded7bd43e974686b3c23bd9d4255b24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:36:55.119269 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67fca3b-444a-4d8b-86ab-12b4c57fa2c1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:36:55.130824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1420/2828] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:36:55 2022] PUT /identity/v3/projects/f26452741ee2411aa990bc713efc2886/users/51c3e51644fc46ab9befd6eee1accda1/roles/aded7bd43e974686b3c23bd9d4255b24 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:37:00.390200 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317727f-dab7-46b4-806c-90b762e5700b None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:00.390790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317727f-dab7-46b4-806c-90b762e5700b None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:00.390790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317727f-dab7-46b4-806c-90b762e5700b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:00.438895 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-5317727f-dab7-46b4-806c-90b762e5700b None None] MFA Rules not processed for user `51c3e51644fc46ab9befd6eee1accda1`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:37:00.440787 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-5317727f-dab7-46b4-806c-90b762e5700b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:00.534815 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1409/2829] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:37:00 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:37:00.548563 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67fca3b-444a-4d8b-86ab-12b4c57fa2c1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:00.550980 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f67fca3b-444a-4d8b-86ab-12b4c57fa2c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:00.617826 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:37:00.620972 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67fca3b-444a-4d8b-86ab-12b4c57fa2c1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:00.625584 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa37c8b-84da-4b14-b810-37aa6cfe0493 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0fecb440c0874ede9f3cf544cfed93ab', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:00.626527 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa37c8b-84da-4b14-b810-37aa6cfe0493 service glance] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:00.627683 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa37c8b-84da-4b14-b810-37aa6cfe0493 service glance] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:00.628506 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa37c8b-84da-4b14-b810-37aa6cfe0493 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:00.696778 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa37c8b-84da-4b14-b810-37aa6cfe0493 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:00.698095 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa37c8b-84da-4b14-b810-37aa6cfe0493 service glance] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:00.709780 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1421/2830] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:37:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:37:00.867924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d73293-6dba-47e2-b80b-fc6d9be0bad9 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:00.868220 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d73293-6dba-47e2-b80b-fc6d9be0bad9 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:00.868504 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d73293-6dba-47e2-b80b-fc6d9be0bad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:00.881129 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:37:00.903755 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-29d73293-6dba-47e2-b80b-fc6d9be0bad9 None None] MFA Rules not processed for user `85626350279c40ea99da93b9ed6994b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:37:00.904985 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-29d73293-6dba-47e2-b80b-fc6d9be0bad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:00.977827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1410/2831] 10.222.0.39 () {58 vars in 1009 bytes} [Thu Jul 28 22:37:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:37:01.025228 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa37c8b-84da-4b14-b810-37aa6cfe0493 service glance] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:01.026739 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6fa37c8b-84da-4b14-b810-37aa6cfe0493 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:01.095059 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa37c8b-84da-4b14-b810-37aa6cfe0493 service glance] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:01.100560 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4c1dd8-5173-432b-ae1a-57905a23891d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9107872f147f6ab8698b67bb545af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:01.101171 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c1dd8-5173-432b-ae1a-57905a23891d service swift] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:01.101453 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c1dd8-5173-432b-ae1a-57905a23891d service swift] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:01.101759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4c1dd8-5173-432b-ae1a-57905a23891d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:01.110214 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c1dd8-5173-432b-ae1a-57905a23891d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:01.111793 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4c1dd8-5173-432b-ae1a-57905a23891d service swift] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:01.120596 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1422/2832] 10.222.0.39 () {60 vars in 1384 bytes} [Thu Jul 28 22:37:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 22:37:03.349035 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d73293-6dba-47e2-b80b-fc6d9be0bad9 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:03.351593 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-29d73293-6dba-47e2-b80b-fc6d9be0bad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:03.455026 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d73293-6dba-47e2-b80b-fc6d9be0bad9 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:03.461798 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e7309-b21a-4dcf-ab2b-d1dbe4e3cc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:03.462601 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e7309-b21a-4dcf-ab2b-d1dbe4e3cc46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:03.462954 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e7309-b21a-4dcf-ab2b-d1dbe4e3cc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:03.463307 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0e7309-b21a-4dcf-ab2b-d1dbe4e3cc46 admin admin] PATH_INFO: `/v3/users/be4c24d2acc444de93cc573dbc096f1d` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:03.464729 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e7309-b21a-4dcf-ab2b-d1dbe4e3cc46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be4c24d2acc444de93cc573dbc096f1d)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:03.466791 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0e7309-b21a-4dcf-ab2b-d1dbe4e3cc46 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:03.543881 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4e0e7309-b21a-4dcf-ab2b-d1dbe4e3cc46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be4c24d2acc444de93cc573dbc096f1d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:03.558564 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4e0e7309-b21a-4dcf-ab2b-d1dbe4e3cc46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be4c24d2acc444de93cc573dbc096f1d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:03.569672 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4e0e7309-b21a-4dcf-ab2b-d1dbe4e3cc46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be4c24d2acc444de93cc573dbc096f1d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:03.570994 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4e0e7309-b21a-4dcf-ab2b-d1dbe4e3cc46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be4c24d2acc444de93cc573dbc096f1d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:03.588297 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4e0e7309-b21a-4dcf-ab2b-d1dbe4e3cc46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be4c24d2acc444de93cc573dbc096f1d'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:03.595429 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1411/2833] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:37:03 2022] DELETE /identity/v3/users/be4c24d2acc444de93cc573dbc096f1d => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:37:03.605822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6877a-a479-4384-ba07-add61f016108 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:03.606308 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6877a-a479-4384-ba07-add61f016108 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:03.606407 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6877a-a479-4384-ba07-add61f016108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:03.650984 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1ab6877a-a479-4384-ba07-add61f016108 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:37:03.653138 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1ab6877a-a479-4384-ba07-add61f016108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:03.781561 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1423/2834] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:37:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:37:03.799431 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e7309-b21a-4dcf-ab2b-d1dbe4e3cc46 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:03.800903 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4e0e7309-b21a-4dcf-ab2b-d1dbe4e3cc46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:03.855651 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:37:03.974930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0e7309-b21a-4dcf-ab2b-d1dbe4e3cc46 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:03.982852 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e5b05-b007-4062-96af-ecc4e93bb900 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:03.985907 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e5b05-b007-4062-96af-ecc4e93bb900 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:03.988039 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e5b05-b007-4062-96af-ecc4e93bb900 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:03.988939 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e5b05-b007-4062-96af-ecc4e93bb900 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:04.005942 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e5b05-b007-4062-96af-ecc4e93bb900 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:04.008087 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e5b05-b007-4062-96af-ecc4e93bb900 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:04.034533 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1412/2835] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:37:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:37:04.617128 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6877a-a479-4384-ba07-add61f016108 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:04.620170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1ab6877a-a479-4384-ba07-add61f016108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:04.643945 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6877a-a479-4384-ba07-add61f016108 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:04.656432 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989618a3-3bce-4640-b018-e995bd705d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:04.657113 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989618a3-3bce-4640-b018-e995bd705d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:04.657396 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989618a3-3bce-4640-b018-e995bd705d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:04.657711 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989618a3-3bce-4640-b018-e995bd705d8e admin admin] PATH_INFO: `/v3/projects/b4beb065ba474c8594da3c4d1072026e` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:04.659106 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989618a3-3bce-4640-b018-e995bd705d8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4beb065ba474c8594da3c4d1072026e)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:04.662708 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989618a3-3bce-4640-b018-e995bd705d8e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:04.714034 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:37:04.734566 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-989618a3-3bce-4640-b018-e995bd705d8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4beb065ba474c8594da3c4d1072026e', 'request_id': 'req-989618a3-3bce-4640-b018-e995bd705d8e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:04.743958 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-989618a3-3bce-4640-b018-e995bd705d8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4beb065ba474c8594da3c4d1072026e', 'request_id': 'req-989618a3-3bce-4640-b018-e995bd705d8e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:04.757295 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-989618a3-3bce-4640-b018-e995bd705d8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4beb065ba474c8594da3c4d1072026e', 'request_id': 'req-989618a3-3bce-4640-b018-e995bd705d8e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:04.774219 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-989618a3-3bce-4640-b018-e995bd705d8e admin admin] The token cache is being invalidate because project b4beb065ba474c8594da3c4d1072026e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:37:04.774557 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-989618a3-3bce-4640-b018-e995bd705d8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:04.775122 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-989618a3-3bce-4640-b018-e995bd705d8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:04.778981 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1424/2836] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:37:04 2022] DELETE /identity/v3/projects/b4beb065ba474c8594da3c4d1072026e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:37:05.178418 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e5b05-b007-4062-96af-ecc4e93bb900 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:05.180397 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0a1e5b05-b007-4062-96af-ecc4e93bb900 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:05.196521 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e5b05-b007-4062-96af-ecc4e93bb900 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:05.263725 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc75feb-c9cf-437d-b30e-0b377c869ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:05.264687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc75feb-c9cf-437d-b30e-0b377c869ae9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:05.265133 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc75feb-c9cf-437d-b30e-0b377c869ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:05.265609 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc75feb-c9cf-437d-b30e-0b377c869ae9 admin admin] PATH_INFO: `/v3/users/5c5b5142c8bf49f58b94defd51697694` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:05.266975 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc75feb-c9cf-437d-b30e-0b377c869ae9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c5b5142c8bf49f58b94defd51697694)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:05.268915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc75feb-c9cf-437d-b30e-0b377c869ae9 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:05.329456 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4bc75feb-c9cf-437d-b30e-0b377c869ae9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c5b5142c8bf49f58b94defd51697694'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:05.338431 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4bc75feb-c9cf-437d-b30e-0b377c869ae9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c5b5142c8bf49f58b94defd51697694'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:05.349068 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4bc75feb-c9cf-437d-b30e-0b377c869ae9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c5b5142c8bf49f58b94defd51697694'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:05.349862 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4bc75feb-c9cf-437d-b30e-0b377c869ae9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c5b5142c8bf49f58b94defd51697694'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:05.369379 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-4bc75feb-c9cf-437d-b30e-0b377c869ae9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c5b5142c8bf49f58b94defd51697694'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:05.383732 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1413/2837] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:37:05 2022] DELETE /identity/v3/users/5c5b5142c8bf49f58b94defd51697694 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:37:05.385445 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21827613-64b4-4939-a7c8-aa9774856a88 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:05.385951 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21827613-64b4-4939-a7c8-aa9774856a88 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:05.386382 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21827613-64b4-4939-a7c8-aa9774856a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:05.416615 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-21827613-64b4-4939-a7c8-aa9774856a88 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:37:05.418166 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-21827613-64b4-4939-a7c8-aa9774856a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:05.509227 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1425/2838] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:37:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:37:05.519797 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc75feb-c9cf-437d-b30e-0b377c869ae9 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:05.522064 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-4bc75feb-c9cf-437d-b30e-0b377c869ae9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:05.577591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:37:05.602292 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc75feb-c9cf-437d-b30e-0b377c869ae9 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:05.609321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c91069-c8df-4831-a0ad-72dd964a3f77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:05.610083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c91069-c8df-4831-a0ad-72dd964a3f77 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:05.610371 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c91069-c8df-4831-a0ad-72dd964a3f77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:05.610651 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c91069-c8df-4831-a0ad-72dd964a3f77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:05.626858 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c91069-c8df-4831-a0ad-72dd964a3f77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:05.628988 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c91069-c8df-4831-a0ad-72dd964a3f77 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:05.657288 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1414/2839] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:37:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:37:06.018190 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21827613-64b4-4939-a7c8-aa9774856a88 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:06.021797 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-21827613-64b4-4939-a7c8-aa9774856a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:06.049273 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21827613-64b4-4939-a7c8-aa9774856a88 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:06.057302 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df62ab-5679-46aa-aeaf-7adf3763085d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:06.058068 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df62ab-5679-46aa-aeaf-7adf3763085d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:06.058359 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df62ab-5679-46aa-aeaf-7adf3763085d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:06.058641 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df62ab-5679-46aa-aeaf-7adf3763085d admin admin] PATH_INFO: `/v3/projects/8c94b7f889f847a1aee252d7e7583961` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:06.060833 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1df62ab-5679-46aa-aeaf-7adf3763085d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c94b7f889f847a1aee252d7e7583961)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:06.063023 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1df62ab-5679-46aa-aeaf-7adf3763085d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:06.134533 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d1df62ab-5679-46aa-aeaf-7adf3763085d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c94b7f889f847a1aee252d7e7583961', 'request_id': 'req-d1df62ab-5679-46aa-aeaf-7adf3763085d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:06.141301 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d1df62ab-5679-46aa-aeaf-7adf3763085d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c94b7f889f847a1aee252d7e7583961', 'request_id': 'req-d1df62ab-5679-46aa-aeaf-7adf3763085d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:06.149505 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d1df62ab-5679-46aa-aeaf-7adf3763085d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c94b7f889f847a1aee252d7e7583961', 'request_id': 'req-d1df62ab-5679-46aa-aeaf-7adf3763085d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:06.170594 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d1df62ab-5679-46aa-aeaf-7adf3763085d admin admin] The token cache is being invalidate because project 8c94b7f889f847a1aee252d7e7583961 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:37:06.170846 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d1df62ab-5679-46aa-aeaf-7adf3763085d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:06.171558 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d1df62ab-5679-46aa-aeaf-7adf3763085d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:06.173429 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1426/2840] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:37:06 2022] DELETE /identity/v3/projects/8c94b7f889f847a1aee252d7e7583961 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:37:06.180748 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16085252-ca05-424b-a09b-8adb192b5cbc None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:06.181239 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16085252-ca05-424b-a09b-8adb192b5cbc None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:06.181239 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16085252-ca05-424b-a09b-8adb192b5cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:06.207511 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-16085252-ca05-424b-a09b-8adb192b5cbc None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:37:06.208762 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-16085252-ca05-424b-a09b-8adb192b5cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:06.282662 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1415/2841] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:37:06.291169 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419db8f-2884-4cb8-bb27-e0aed963dbf1 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:06.291573 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419db8f-2884-4cb8-bb27-e0aed963dbf1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:06.291919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7419db8f-2884-4cb8-bb27-e0aed963dbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:06.322654 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7419db8f-2884-4cb8-bb27-e0aed963dbf1 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:37:06.323982 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7419db8f-2884-4cb8-bb27-e0aed963dbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:06.334894 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1427/2842] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:37:06.339054 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16085252-ca05-424b-a09b-8adb192b5cbc None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:06.351070 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16085252-ca05-424b-a09b-8adb192b5cbc None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:06.356574 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ad354-3c30-4e20-9024-7ca46d72e0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:06.357374 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ad354-3c30-4e20-9024-7ca46d72e0ce admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:06.357900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ad354-3c30-4e20-9024-7ca46d72e0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:06.358177 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ad354-3c30-4e20-9024-7ca46d72e0ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:06.358883 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17ad354-3c30-4e20-9024-7ca46d72e0ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 28 22:37:06.359195 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17ad354-3c30-4e20-9024-7ca46d72e0ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:06.360814 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17ad354-3c30-4e20-9024-7ca46d72e0ce admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:06.378270 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1416/2843] 10.222.0.39 () {58 vars in 1153 bytes} [Thu Jul 28 22:37:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 28 22:37:06.380582 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419db8f-2884-4cb8-bb27-e0aed963dbf1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:06.390221 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7419db8f-2884-4cb8-bb27-e0aed963dbf1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:06.395576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b9d88-8b7b-43c1-9dfc-d18d21c8a75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:06.396291 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b9d88-8b7b-43c1-9dfc-d18d21c8a75a admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:06.396637 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b9d88-8b7b-43c1-9dfc-d18d21c8a75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:06.396970 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4b9d88-8b7b-43c1-9dfc-d18d21c8a75a admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:06.398507 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b9d88-8b7b-43c1-9dfc-d18d21c8a75a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:06.399673 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4b9d88-8b7b-43c1-9dfc-d18d21c8a75a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:06.435511 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1428/2844] 10.222.0.39 () {60 vars in 1153 bytes} [Thu Jul 28 22:37:06 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 28 22:37:06.439210 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ad354-3c30-4e20-9024-7ca46d72e0ce admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:06.450576 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17ad354-3c30-4e20-9024-7ca46d72e0ce admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:06.537645 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65905a54-f733-4cf9-8b42-e4218316ab14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:06.538390 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65905a54-f733-4cf9-8b42-e4218316ab14 admin admin] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:06.538741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65905a54-f733-4cf9-8b42-e4218316ab14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:06.539115 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65905a54-f733-4cf9-8b42-e4218316ab14 admin admin] PATH_INFO: `/v3/users` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:06.539883 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65905a54-f733-4cf9-8b42-e4218316ab14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:06.541897 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65905a54-f733-4cf9-8b42-e4218316ab14 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:06.566104 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1417/2845] 10.222.0.39 () {60 vars in 1144 bytes} [Thu Jul 28 22:37:06 2022] POST /identity/v3/users => generated 287 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 28 22:37:06.570984 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b9d88-8b7b-43c1-9dfc-d18d21c8a75a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:06.584727 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4b9d88-8b7b-43c1-9dfc-d18d21c8a75a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:06.590147 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e9deb-ab6e-40d7-947c-78e7cd8b6418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:06.591069 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e9deb-ab6e-40d7-947c-78e7cd8b6418 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:06.591551 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e9deb-ab6e-40d7-947c-78e7cd8b6418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:06.592040 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e9deb-ab6e-40d7-947c-78e7cd8b6418 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:06.592754 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e9deb-ab6e-40d7-947c-78e7cd8b6418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:06.594412 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e9deb-ab6e-40d7-947c-78e7cd8b6418 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:06.605324 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:37:06.619096 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1429/2846] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 28 22:37:06.623336 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65905a54-f733-4cf9-8b42-e4218316ab14 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:06.635545 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65905a54-f733-4cf9-8b42-e4218316ab14 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:06.641157 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e85f053-f147-4745-8156-15bf75b23e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:06.641915 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85f053-f147-4745-8156-15bf75b23e2d admin admin] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:06.642230 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85f053-f147-4745-8156-15bf75b23e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:06.642505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e85f053-f147-4745-8156-15bf75b23e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:06.643207 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e85f053-f147-4745-8156-15bf75b23e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:06.644851 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e85f053-f147-4745-8156-15bf75b23e2d admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:06.657957 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1418/2847] 10.222.0.39 () {58 vars in 1122 bytes} [Thu Jul 28 22:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 28 22:37:06.662238 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e9deb-ab6e-40d7-947c-78e7cd8b6418 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:06.672835 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e9deb-ab6e-40d7-947c-78e7cd8b6418 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:06.678875 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bfa12-c6db-47d0-903a-58012d842f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:06.679562 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bfa12-c6db-47d0-903a-58012d842f18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:06.679848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bfa12-c6db-47d0-903a-58012d842f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:06.680118 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3bfa12-c6db-47d0-903a-58012d842f18 admin admin] PATH_INFO: `/v3/projects/314a6ead5fe74aeda4d817081b9571eb/users/bb6d5a1ea4f949be98f89a108fa25d53/roles/4912a35790fe4d4297540156083d81d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:06.698671 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bfa12-c6db-47d0-903a-58012d842f18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=314a6ead5fe74aeda4d817081b9571eb, user_id=bb6d5a1ea4f949be98f89a108fa25d53, role_id=4912a35790fe4d4297540156083d81d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:06.701299 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3bfa12-c6db-47d0-903a-58012d842f18 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:06.719508 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1430/2848] 10.222.0.39 () {60 vars in 1483 bytes} [Thu Jul 28 22:37:06 2022] PUT /identity/v3/projects/314a6ead5fe74aeda4d817081b9571eb/users/bb6d5a1ea4f949be98f89a108fa25d53/roles/4912a35790fe4d4297540156083d81d1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:37:06.732020 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a1461-ed41-4881-b503-986bff58c318 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:06.732305 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a1461-ed41-4881-b503-986bff58c318 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:06.732581 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855a1461-ed41-4881-b503-986bff58c318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:06.769948 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-855a1461-ed41-4881-b503-986bff58c318 None None] MFA Rules not processed for user `bb6d5a1ea4f949be98f89a108fa25d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:37:06.771190 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-855a1461-ed41-4881-b503-986bff58c318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:06.820083 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:37:06.873796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1419/2849] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:37:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:37:06.888695 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bfa12-c6db-47d0-903a-58012d842f18 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:06.890493 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-eb3bfa12-c6db-47d0-903a-58012d842f18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:06.971937 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3bfa12-c6db-47d0-903a-58012d842f18 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:06.975938 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c541f5-d675-4fc7-89ca-0dea24444edf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:06.976646 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c541f5-d675-4fc7-89ca-0dea24444edf service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:06.977014 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c541f5-d675-4fc7-89ca-0dea24444edf service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:06.977348 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c541f5-d675-4fc7-89ca-0dea24444edf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:06.989475 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c541f5-d675-4fc7-89ca-0dea24444edf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:06.991038 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c541f5-d675-4fc7-89ca-0dea24444edf service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:07.009671 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1431/2850] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:37:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:37:08.476312 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d903eb-e980-4b80-8f6b-9201b6f950df None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:08.476908 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d903eb-e980-4b80-8f6b-9201b6f950df None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:08.477012 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d903eb-e980-4b80-8f6b-9201b6f950df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:08.497193 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-62d903eb-e980-4b80-8f6b-9201b6f950df None None] MFA Rules not processed for user `bb6d5a1ea4f949be98f89a108fa25d53`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:37:08.498563 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-62d903eb-e980-4b80-8f6b-9201b6f950df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:08.508523 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1420/2851] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:37:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:37:10.894809 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa3fd7-cffb-48bb-9cef-476e9f13f504 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:10.895785 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa3fd7-cffb-48bb-9cef-476e9f13f504 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:10.896134 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa3fd7-cffb-48bb-9cef-476e9f13f504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:10.916504 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-cdfa3fd7-cffb-48bb-9cef-476e9f13f504 None None] MFA Rules not processed for user `2e64644dc881469e907a6f6669999757`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:37:10.917768 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cdfa3fd7-cffb-48bb-9cef-476e9f13f504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:10.977354 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1432/2852] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:37:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:37:12.853111 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d903eb-e980-4b80-8f6b-9201b6f950df None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:12.854657 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-62d903eb-e980-4b80-8f6b-9201b6f950df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:12.935811 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d903eb-e980-4b80-8f6b-9201b6f950df None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:12.940783 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455be8b5-ed95-484d-8ca9-7cd40f5db4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:12.941460 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455be8b5-ed95-484d-8ca9-7cd40f5db4cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:12.943174 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455be8b5-ed95-484d-8ca9-7cd40f5db4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:12.943502 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455be8b5-ed95-484d-8ca9-7cd40f5db4cc admin admin] PATH_INFO: `/v3/users/f44c045f18c04a29a811a313b65ea710` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:12.944796 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455be8b5-ed95-484d-8ca9-7cd40f5db4cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=f44c045f18c04a29a811a313b65ea710)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:12.946723 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455be8b5-ed95-484d-8ca9-7cd40f5db4cc admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:13.001293 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-455be8b5-ed95-484d-8ca9-7cd40f5db4cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f44c045f18c04a29a811a313b65ea710'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:13.011404 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-455be8b5-ed95-484d-8ca9-7cd40f5db4cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f44c045f18c04a29a811a313b65ea710'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:13.022116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-455be8b5-ed95-484d-8ca9-7cd40f5db4cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f44c045f18c04a29a811a313b65ea710'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:13.023102 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-455be8b5-ed95-484d-8ca9-7cd40f5db4cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f44c045f18c04a29a811a313b65ea710'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:13.038223 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-455be8b5-ed95-484d-8ca9-7cd40f5db4cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f44c045f18c04a29a811a313b65ea710'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:13.044286 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1421/2853] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:37:12 2022] DELETE /identity/v3/users/f44c045f18c04a29a811a313b65ea710 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:37:13.048626 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb437e2-81e7-45da-a48c-411a53e68ee5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:13.049112 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb437e2-81e7-45da-a48c-411a53e68ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:13.049216 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb437e2-81e7-45da-a48c-411a53e68ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:13.070010 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-dbb437e2-81e7-45da-a48c-411a53e68ee5 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:37:13.071487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-dbb437e2-81e7-45da-a48c-411a53e68ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:13.133691 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1433/2854] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:37:13.147226 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455be8b5-ed95-484d-8ca9-7cd40f5db4cc admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:13.148844 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-455be8b5-ed95-484d-8ca9-7cd40f5db4cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:13.185559 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2adfa9-c2db-4476-8d9d-f801875ca877 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:13.185843 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2adfa9-c2db-4476-8d9d-f801875ca877 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:13.186177 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2adfa9-c2db-4476-8d9d-f801875ca877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:13.209015 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5e2adfa9-c2db-4476-8d9d-f801875ca877 None None] MFA Rules not processed for user `85626350279c40ea99da93b9ed6994b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:37:13.210277 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5e2adfa9-c2db-4476-8d9d-f801875ca877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:13.231987 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:37:13.235453 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:37:13.241437 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455be8b5-ed95-484d-8ca9-7cd40f5db4cc admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:13.260568 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973438b-d077-4521-ab79-4f0ea02bdfff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:13.261321 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973438b-d077-4521-ab79-4f0ea02bdfff service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:13.261739 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973438b-d077-4521-ab79-4f0ea02bdfff service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:13.262122 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973438b-d077-4521-ab79-4f0ea02bdfff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:13.272824 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3973438b-d077-4521-ab79-4f0ea02bdfff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:13.274462 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3973438b-d077-4521-ab79-4f0ea02bdfff service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:13.280454 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1434/2855] 10.222.0.39 () {58 vars in 1009 bytes} [Thu Jul 28 22:37:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:37:13.287401 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1422/2856] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:37:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:37:13.294795 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2adfa9-c2db-4476-8d9d-f801875ca877 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:13.296302 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5e2adfa9-c2db-4476-8d9d-f801875ca877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:13.390467 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2adfa9-c2db-4476-8d9d-f801875ca877 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:13.395944 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e4389-4b16-4153-87ed-54bf2d1b5a74 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9107872f147f6ab8698b67bb545af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:13.396577 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e4389-4b16-4153-87ed-54bf2d1b5a74 service swift] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:13.396864 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e4389-4b16-4153-87ed-54bf2d1b5a74 service swift] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:13.397141 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e4389-4b16-4153-87ed-54bf2d1b5a74 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:13.406651 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e4389-4b16-4153-87ed-54bf2d1b5a74 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:13.408242 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e4389-4b16-4153-87ed-54bf2d1b5a74 service swift] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:13.418889 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1435/2857] 10.222.0.39 () {60 vars in 1384 bytes} [Thu Jul 28 22:37:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 22:37:13.615662 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973438b-d077-4521-ab79-4f0ea02bdfff service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:13.617338 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3973438b-d077-4521-ab79-4f0ea02bdfff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:13.646260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973438b-d077-4521-ab79-4f0ea02bdfff service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:13.652329 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb46ce2-17f6-419b-acbb-265b70459b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:13.653114 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb46ce2-17f6-419b-acbb-265b70459b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:13.653479 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb46ce2-17f6-419b-acbb-265b70459b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:13.653779 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb46ce2-17f6-419b-acbb-265b70459b24 admin admin] PATH_INFO: `/v3/projects/c5b2013a802a4661952c1f1260d29e55` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:13.655176 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb46ce2-17f6-419b-acbb-265b70459b24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5b2013a802a4661952c1f1260d29e55)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:13.657260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb46ce2-17f6-419b-acbb-265b70459b24 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:13.723008 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-eeb46ce2-17f6-419b-acbb-265b70459b24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5b2013a802a4661952c1f1260d29e55', 'request_id': 'req-eeb46ce2-17f6-419b-acbb-265b70459b24'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:13.736731 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-eeb46ce2-17f6-419b-acbb-265b70459b24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5b2013a802a4661952c1f1260d29e55', 'request_id': 'req-eeb46ce2-17f6-419b-acbb-265b70459b24'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:13.746706 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-eeb46ce2-17f6-419b-acbb-265b70459b24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5b2013a802a4661952c1f1260d29e55', 'request_id': 'req-eeb46ce2-17f6-419b-acbb-265b70459b24'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:13.762141 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-eeb46ce2-17f6-419b-acbb-265b70459b24 admin admin] The token cache is being invalidate because project c5b2013a802a4661952c1f1260d29e55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:37:13.762444 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-eeb46ce2-17f6-419b-acbb-265b70459b24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:13.764075 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-eeb46ce2-17f6-419b-acbb-265b70459b24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:13.766194 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1423/2858] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:37:13 2022] DELETE /identity/v3/projects/c5b2013a802a4661952c1f1260d29e55 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:37:23.415479 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=46 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:37:23.757717 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=31,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:37:28.478950 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e75b0-d4e5-4974-9a5c-ac49dfe3db17 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:28.479435 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e75b0-d4e5-4974-9a5c-ac49dfe3db17 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:28.479535 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560e75b0-d4e5-4974-9a5c-ac49dfe3db17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:28.498967 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-560e75b0-d4e5-4974-9a5c-ac49dfe3db17 None None] MFA Rules not processed for user `85626350279c40ea99da93b9ed6994b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:37:28.500157 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-560e75b0-d4e5-4974-9a5c-ac49dfe3db17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:28.554251 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1436/2859] 10.222.0.39 () {58 vars in 1009 bytes} [Thu Jul 28 22:37:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:37:28.566117 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb46ce2-17f6-419b-acbb-265b70459b24 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:28.567529 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-eeb46ce2-17f6-419b-acbb-265b70459b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:28.637788 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb46ce2-17f6-419b-acbb-265b70459b24 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:28.642815 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bef318-bf09-4248-8daa-082b31d8bfca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9107872f147f6ab8698b67bb545af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:28.643433 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bef318-bf09-4248-8daa-082b31d8bfca service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:28.643718 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bef318-bf09-4248-8daa-082b31d8bfca service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:28.643998 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bef318-bf09-4248-8daa-082b31d8bfca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:28.652073 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bef318-bf09-4248-8daa-082b31d8bfca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:28.653695 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bef318-bf09-4248-8daa-082b31d8bfca service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:28.661633 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1424/2860] 10.222.0.39 () {60 vars in 1384 bytes} [Thu Jul 28 22:37:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 22:37:29.183798 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55375b-7cda-40be-b4e2-8a89572002f9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:29.184085 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55375b-7cda-40be-b4e2-8a89572002f9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:29.184364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f55375b-7cda-40be-b4e2-8a89572002f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:29.203781 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5f55375b-7cda-40be-b4e2-8a89572002f9 None None] MFA Rules not processed for user `85626350279c40ea99da93b9ed6994b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:37:29.204971 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5f55375b-7cda-40be-b4e2-8a89572002f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:29.212640 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1437/2861] 10.222.0.39 () {58 vars in 967 bytes} [Thu Jul 28 22:37:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:37:29.225963 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bef318-bf09-4248-8daa-082b31d8bfca service swift] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:29.236445 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bef318-bf09-4248-8daa-082b31d8bfca service swift] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:29.241982 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a8671-7b40-440f-b882-3e56cf2ae873 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9107872f147f6ab8698b67bb545af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:29.242591 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a8671-7b40-440f-b882-3e56cf2ae873 service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:29.242901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a8671-7b40-440f-b882-3e56cf2ae873 service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:29.243181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a8671-7b40-440f-b882-3e56cf2ae873 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:29.251320 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a8671-7b40-440f-b882-3e56cf2ae873 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:29.252901 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a8671-7b40-440f-b882-3e56cf2ae873 service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:29.261659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1425/2862] 10.222.0.39 () {60 vars in 1384 bytes} [Thu Jul 28 22:37:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 28 22:37:29.490723 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2c1b2-3765-4faa-983e-6961428478da None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:29.491750 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2c1b2-3765-4faa-983e-6961428478da None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:29.492207 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2c1b2-3765-4faa-983e-6961428478da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:29.513364 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4db2c1b2-3765-4faa-983e-6961428478da None None] MFA Rules not processed for user `85626350279c40ea99da93b9ed6994b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:37:29.515460 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4db2c1b2-3765-4faa-983e-6961428478da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:29.524328 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1438/2863] 10.222.0.39 () {58 vars in 1009 bytes} [Thu Jul 28 22:37:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 28 22:37:29.540325 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a8671-7b40-440f-b882-3e56cf2ae873 service swift] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:29.557886 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a8671-7b40-440f-b882-3e56cf2ae873 service swift] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:29.563006 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04fd5b-1061-41c9-9f16-578143bed997 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9107872f147f6ab8698b67bb545af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:29.563629 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04fd5b-1061-41c9-9f16-578143bed997 service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:29.563910 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04fd5b-1061-41c9-9f16-578143bed997 service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:29.564182 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04fd5b-1061-41c9-9f16-578143bed997 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:29.577460 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04fd5b-1061-41c9-9f16-578143bed997 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:29.579157 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f04fd5b-1061-41c9-9f16-578143bed997 service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:29.588523 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1426/2864] 10.222.0.39 () {60 vars in 1384 bytes} [Thu Jul 28 22:37:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 22:37:29.916805 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32d41a-0d1d-49f3-87e5-b0ae5d5ef9ee None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:29.917344 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32d41a-0d1d-49f3-87e5-b0ae5d5ef9ee None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:29.917678 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd32d41a-0d1d-49f3-87e5-b0ae5d5ef9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:29.936974 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-cd32d41a-0d1d-49f3-87e5-b0ae5d5ef9ee None None] MFA Rules not processed for user `85626350279c40ea99da93b9ed6994b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:37:29.938293 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cd32d41a-0d1d-49f3-87e5-b0ae5d5ef9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:29.946376 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1439/2865] 10.222.0.39 () {58 vars in 967 bytes} [Thu Jul 28 22:37:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:37:29.958833 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04fd5b-1061-41c9-9f16-578143bed997 service swift] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:29.970037 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04fd5b-1061-41c9-9f16-578143bed997 service swift] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:29.975827 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf6cd16-3481-40c8-92cd-f4d4ceda4204 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9107872f147f6ab8698b67bb545af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:29.976505 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6cd16-3481-40c8-92cd-f4d4ceda4204 service swift] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:29.976902 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6cd16-3481-40c8-92cd-f4d4ceda4204 service swift] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:29.977296 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf6cd16-3481-40c8-92cd-f4d4ceda4204 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:29.985561 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf6cd16-3481-40c8-92cd-f4d4ceda4204 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:29.987133 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf6cd16-3481-40c8-92cd-f4d4ceda4204 service swift] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:29.995993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1427/2866] 10.222.0.39 () {60 vars in 1384 bytes} [Thu Jul 28 22:37:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 22:37:32.083592 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32d41a-0d1d-49f3-87e5-b0ae5d5ef9ee None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:32.085062 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cd32d41a-0d1d-49f3-87e5-b0ae5d5ef9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:32.151220 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd32d41a-0d1d-49f3-87e5-b0ae5d5ef9ee None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:32.155742 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c303af-6488-4e04-8658-25cfecf23603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:32.156381 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c303af-6488-4e04-8658-25cfecf23603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:32.156661 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c303af-6488-4e04-8658-25cfecf23603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:32.156933 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c303af-6488-4e04-8658-25cfecf23603 admin admin] PATH_INFO: `/v3/users/bb6d5a1ea4f949be98f89a108fa25d53` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:32.158132 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c303af-6488-4e04-8658-25cfecf23603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb6d5a1ea4f949be98f89a108fa25d53)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:32.160052 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c303af-6488-4e04-8658-25cfecf23603 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:32.206274 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d3c303af-6488-4e04-8658-25cfecf23603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb6d5a1ea4f949be98f89a108fa25d53'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:32.213948 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d3c303af-6488-4e04-8658-25cfecf23603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb6d5a1ea4f949be98f89a108fa25d53'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:32.214694 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d3c303af-6488-4e04-8658-25cfecf23603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb6d5a1ea4f949be98f89a108fa25d53'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:32.215383 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d3c303af-6488-4e04-8658-25cfecf23603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb6d5a1ea4f949be98f89a108fa25d53'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:32.227922 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d3c303af-6488-4e04-8658-25cfecf23603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb6d5a1ea4f949be98f89a108fa25d53'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:32.240905 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1440/2867] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:37:32 2022] DELETE /identity/v3/users/bb6d5a1ea4f949be98f89a108fa25d53 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:37:32.245629 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8ad28-1f17-44ce-b165-fe3bf9cdd812 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:32.246089 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8ad28-1f17-44ce-b165-fe3bf9cdd812 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:32.246294 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e8ad28-1f17-44ce-b165-fe3bf9cdd812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:32.264818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-27e8ad28-1f17-44ce-b165-fe3bf9cdd812 None None] MFA Rules not processed for user `0b20c2888f9642efb7858178880ed9a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:37:32.266043 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-27e8ad28-1f17-44ce-b165-fe3bf9cdd812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:32.324110 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1428/2868] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:37:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:37:32.333694 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c303af-6488-4e04-8658-25cfecf23603 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:32.335616 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d3c303af-6488-4e04-8658-25cfecf23603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:32.342170 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:37:32.397056 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c303af-6488-4e04-8658-25cfecf23603 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:32.401094 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82c518-189f-4c22-85c0-6b4ab653be1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:32.401728 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82c518-189f-4c22-85c0-6b4ab653be1e service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:32.402014 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82c518-189f-4c22-85c0-6b4ab653be1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:32.402285 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82c518-189f-4c22-85c0-6b4ab653be1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:32.411155 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82c518-189f-4c22-85c0-6b4ab653be1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:32.413657 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82c518-189f-4c22-85c0-6b4ab653be1e service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:32.422997 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1441/2869] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:37:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:37:32.595283 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8ad28-1f17-44ce-b165-fe3bf9cdd812 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:32.596922 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-27e8ad28-1f17-44ce-b165-fe3bf9cdd812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:32.610116 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e8ad28-1f17-44ce-b165-fe3bf9cdd812 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:32.615246 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c8507-8c48-42f4-b7d8-37cc3ad01041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:32.615941 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c8507-8c48-42f4-b7d8-37cc3ad01041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:32.616286 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c8507-8c48-42f4-b7d8-37cc3ad01041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:32.616492 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c8507-8c48-42f4-b7d8-37cc3ad01041 admin admin] PATH_INFO: `/v3/projects/314a6ead5fe74aeda4d817081b9571eb` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:32.617710 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c8507-8c48-42f4-b7d8-37cc3ad01041 admin admin] RBAC: Authorizing `identity:delete_project(project_id=314a6ead5fe74aeda4d817081b9571eb)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:32.619638 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c8507-8c48-42f4-b7d8-37cc3ad01041 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:32.665794 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:37:32.669023 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b45c8507-8c48-42f4-b7d8-37cc3ad01041 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '314a6ead5fe74aeda4d817081b9571eb', 'request_id': 'req-b45c8507-8c48-42f4-b7d8-37cc3ad01041'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:32.676329 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b45c8507-8c48-42f4-b7d8-37cc3ad01041 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '314a6ead5fe74aeda4d817081b9571eb', 'request_id': 'req-b45c8507-8c48-42f4-b7d8-37cc3ad01041'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:32.681940 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b45c8507-8c48-42f4-b7d8-37cc3ad01041 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '314a6ead5fe74aeda4d817081b9571eb', 'request_id': 'req-b45c8507-8c48-42f4-b7d8-37cc3ad01041'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:32.692712 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b45c8507-8c48-42f4-b7d8-37cc3ad01041 admin admin] The token cache is being invalidate because project 314a6ead5fe74aeda4d817081b9571eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:37:32.692999 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b45c8507-8c48-42f4-b7d8-37cc3ad01041 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:32.694310 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b45c8507-8c48-42f4-b7d8-37cc3ad01041 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:37:32.695818 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1429/2870] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:37:32 2022] DELETE /identity/v3/projects/314a6ead5fe74aeda4d817081b9571eb => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:37:42.306524 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82c518-189f-4c22-85c0-6b4ab653be1e service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:37:42.315094 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82c518-189f-4c22-85c0-6b4ab653be1e service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:37:42.362952 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb41c9f-4337-44cc-90c2-acb50989eaf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:37:42.363562 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb41c9f-4337-44cc-90c2-acb50989eaf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:37:42.363844 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb41c9f-4337-44cc-90c2-acb50989eaf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:37:42.364119 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb41c9f-4337-44cc-90c2-acb50989eaf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:37:42.365927 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1cb41c9f-4337-44cc-90c2-acb50989eaf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:37:42.426937 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb41c9f-4337-44cc-90c2-acb50989eaf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:37:42.428619 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb41c9f-4337-44cc-90c2-acb50989eaf7 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:37:42.439737 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1442/2871] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:37:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:37:42.689806 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:37:52.435074 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=74 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:38:08.621612 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c8507-8c48-42f4-b7d8-37cc3ad01041 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:38:08.623031 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-b45c8507-8c48-42f4-b7d8-37cc3ad01041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:38:08.679002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c8507-8c48-42f4-b7d8-37cc3ad01041 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:38:08.683206 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d21c49-1d0e-477c-b65e-c8def5ac3b8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4aa320b18ce43c3acfece4546fea0f1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:38:08.683821 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d21c49-1d0e-477c-b65e-c8def5ac3b8a service cinder] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:38:08.684100 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d21c49-1d0e-477c-b65e-c8def5ac3b8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:38:08.684373 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d21c49-1d0e-477c-b65e-c8def5ac3b8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:38:08.686170 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-d4d21c49-1d0e-477c-b65e-c8def5ac3b8a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:38:08.746207 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d21c49-1d0e-477c-b65e-c8def5ac3b8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:38:08.747656 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d21c49-1d0e-477c-b65e-c8def5ac3b8a service cinder] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:38:08.757973 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1430/2872] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:38:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:38:10.067650 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb41c9f-4337-44cc-90c2-acb50989eaf7 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:38:10.069281 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1cb41c9f-4337-44cc-90c2-acb50989eaf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:38:10.084744 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb41c9f-4337-44cc-90c2-acb50989eaf7 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:38:10.091206 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7707cccf-a329-473b-9fb2-86f302a3bc17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:38:10.091827 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707cccf-a329-473b-9fb2-86f302a3bc17 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:38:10.092105 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707cccf-a329-473b-9fb2-86f302a3bc17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:38:10.092378 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7707cccf-a329-473b-9fb2-86f302a3bc17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:38:10.094165 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7707cccf-a329-473b-9fb2-86f302a3bc17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:38:10.177701 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707cccf-a329-473b-9fb2-86f302a3bc17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:38:10.179126 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7707cccf-a329-473b-9fb2-86f302a3bc17 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:38:10.200191 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1443/2873] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:38:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 28 22:38:11.392011 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d21c49-1d0e-477c-b65e-c8def5ac3b8a service cinder] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:38:11.402146 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d21c49-1d0e-477c-b65e-c8def5ac3b8a service cinder] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:38:11.405817 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f0a52-b784-4619-a3a8-b102e318a856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:38:11.406376 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f0a52-b784-4619-a3a8-b102e318a856 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:38:11.406659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f0a52-b784-4619-a3a8-b102e318a856 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:38:11.406935 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6f0a52-b784-4619-a3a8-b102e318a856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:38:11.408747 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0d6f0a52-b784-4619-a3a8-b102e318a856 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:38:11.424354 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f0a52-b784-4619-a3a8-b102e318a856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:38:11.425837 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6f0a52-b784-4619-a3a8-b102e318a856 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:38:11.438232 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1431/2874] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:38:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:38:16.641366 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4caa7fa-968b-402d-bf3b-dba76dfb8d6d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:38:16.641867 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4caa7fa-968b-402d-bf3b-dba76dfb8d6d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:38:16.641989 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4caa7fa-968b-402d-bf3b-dba76dfb8d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:38:16.661385 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c4caa7fa-968b-402d-bf3b-dba76dfb8d6d None None] MFA Rules not processed for user `32eadc02f9224eb28303aab25506ef2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:38:16.662571 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c4caa7fa-968b-402d-bf3b-dba76dfb8d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:38:16.670560 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1444/2875] 10.222.0.39 () {58 vars in 958 bytes} [Thu Jul 28 22:38:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:38:21.433146 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=54 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:38:26.656033 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=46 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:38:32.911878 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f0a52-b784-4619-a3a8-b102e318a856 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:38:32.913280 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-0d6f0a52-b784-4619-a3a8-b102e318a856 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:38:32.924691 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6f0a52-b784-4619-a3a8-b102e318a856 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:38:32.928844 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c4ddf-c38b-4600-a49a-aee5f0fbbbb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:38:32.929446 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c4ddf-c38b-4600-a49a-aee5f0fbbbb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:38:32.929755 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c4ddf-c38b-4600-a49a-aee5f0fbbbb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:38:32.930028 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c4ddf-c38b-4600-a49a-aee5f0fbbbb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:38:32.932602 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-839c4ddf-c38b-4600-a49a-aee5f0fbbbb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:38:32.995084 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c4ddf-c38b-4600-a49a-aee5f0fbbbb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:38:32.996525 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c4ddf-c38b-4600-a49a-aee5f0fbbbb9 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:38:33.006612 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1432/2876] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:38:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:38:43.001879 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=30 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:39:00.532750 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4caa7fa-968b-402d-bf3b-dba76dfb8d6d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:39:00.534430 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c4caa7fa-968b-402d-bf3b-dba76dfb8d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:39:00.563547 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4caa7fa-968b-402d-bf3b-dba76dfb8d6d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:39:00.569943 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfcefc-bcee-429b-a2b0-fc797973b0bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:39:00.570570 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfcefc-bcee-429b-a2b0-fc797973b0bb service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:39:00.570848 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfcefc-bcee-429b-a2b0-fc797973b0bb service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:39:00.571119 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfcefc-bcee-429b-a2b0-fc797973b0bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:39:00.573245 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3dcfcefc-bcee-429b-a2b0-fc797973b0bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:39:00.588658 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfcefc-bcee-429b-a2b0-fc797973b0bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:39:00.590340 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfcefc-bcee-429b-a2b0-fc797973b0bb service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:39:00.607763 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1445/2877] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:39:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:39:00.690557 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c4ddf-c38b-4600-a49a-aee5f0fbbbb9 service neutron] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:39:00.700677 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c4ddf-c38b-4600-a49a-aee5f0fbbbb9 service neutron] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:39:00.706583 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111bf75f-884c-48de-9a24-79a2b653d2c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:39:00.707317 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bf75f-884c-48de-9a24-79a2b653d2c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:39:00.707677 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bf75f-884c-48de-9a24-79a2b653d2c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:39:00.708029 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bf75f-884c-48de-9a24-79a2b653d2c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:39:00.715221 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-111bf75f-884c-48de-9a24-79a2b653d2c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:39:00.733319 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111bf75f-884c-48de-9a24-79a2b653d2c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:39:00.735195 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111bf75f-884c-48de-9a24-79a2b653d2c9 service neutron] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:39:00.750388 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1433/2878] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:39:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:39:07.117199 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfcefc-bcee-429b-a2b0-fc797973b0bb service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:39:07.129649 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfcefc-bcee-429b-a2b0-fc797973b0bb service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:39:07.136244 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc93fb68-d19a-434f-a692-59e652bce6e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77efe8046ea0468cb0c7c58c24fc4e75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:39:07.136836 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93fb68-d19a-434f-a692-59e652bce6e0 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:39:07.137119 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93fb68-d19a-434f-a692-59e652bce6e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:39:07.137398 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93fb68-d19a-434f-a692-59e652bce6e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:39:07.139237 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-dc93fb68-d19a-434f-a692-59e652bce6e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:39:07.155156 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc93fb68-d19a-434f-a692-59e652bce6e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:39:07.156919 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc93fb68-d19a-434f-a692-59e652bce6e0 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:39:07.168182 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1446/2879] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:39:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:39:08.222072 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5c056-7a4b-4fe1-aa90-b5f846224d19 None None] REQUEST_METHOD: `POST` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:39:08.222526 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5c056-7a4b-4fe1-aa90-b5f846224d19 None None] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:39:08.222641 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f5c056-7a4b-4fe1-aa90-b5f846224d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:39:08.272640 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.auth.core [None req-24f5c056-7a4b-4fe1-aa90-b5f846224d19 None None] MFA Rules not processed for user `85626350279c40ea99da93b9ed6994b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 28 22:39:08.273878 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-24f5c056-7a4b-4fe1-aa90-b5f846224d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:39:08.337585 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1434/2880] 10.222.0.39 () {58 vars in 967 bytes} [Thu Jul 28 22:39:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 28 22:39:08.386158 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc93fb68-d19a-434f-a692-59e652bce6e0 service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:39:08.387581 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-dc93fb68-d19a-434f-a692-59e652bce6e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:39:08.474028 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc93fb68-d19a-434f-a692-59e652bce6e0 service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:39:08.479545 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1cd8e-d08f-458d-b698-19496651b72a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3c9107872f147f6ab8698b67bb545af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:39:08.480151 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1cd8e-d08f-458d-b698-19496651b72a service swift] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:39:08.480428 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1cd8e-d08f-458d-b698-19496651b72a service swift] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:39:08.480700 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f1cd8e-d08f-458d-b698-19496651b72a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:39:08.488796 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1cd8e-d08f-458d-b698-19496651b72a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:39:08.490402 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f1cd8e-d08f-458d-b698-19496651b72a service swift] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:39:08.498843 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1447/2881] 10.222.0.39 () {60 vars in 1384 bytes} [Thu Jul 28 22:39:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 28 22:39:11.915687 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5c056-7a4b-4fe1-aa90-b5f846224d19 None None] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:39:11.917203 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-24f5c056-7a4b-4fe1-aa90-b5f846224d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:39:11.996946 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f5c056-7a4b-4fe1-aa90-b5f846224d19 None None] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:39:12.002127 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62ae32-9f6c-477b-b12f-a9c2a666e914 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe003cd7bfbe4b92b1d64d300c3e7d8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:39:12.002759 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62ae32-9f6c-477b-b12f-a9c2a666e914 service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:39:12.003046 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62ae32-9f6c-477b-b12f-a9c2a666e914 service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:39:12.003326 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a62ae32-9f6c-477b-b12f-a9c2a666e914 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:39:12.005372 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-1a62ae32-9f6c-477b-b12f-a9c2a666e914 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:39:12.018267 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62ae32-9f6c-477b-b12f-a9c2a666e914 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:39:12.019837 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a62ae32-9f6c-477b-b12f-a9c2a666e914 service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:39:12.030659 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1435/2882] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:39:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:39:14.921822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1cd8e-d08f-458d-b698-19496651b72a service swift] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:39:14.923358 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-93f1cd8e-d08f-458d-b698-19496651b72a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:39:14.995211 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f1cd8e-d08f-458d-b698-19496651b72a service swift] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:39:15.000255 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d141d12-505e-47f9-ab73-cc904ea7a9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:39:15.000920 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d141d12-505e-47f9-ab73-cc904ea7a9a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:39:15.001208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d141d12-505e-47f9-ab73-cc904ea7a9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:39:15.001489 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d141d12-505e-47f9-ab73-cc904ea7a9a7 admin admin] PATH_INFO: `/v3/users/2e64644dc881469e907a6f6669999757` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:39:15.002824 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d141d12-505e-47f9-ab73-cc904ea7a9a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e64644dc881469e907a6f6669999757)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:39:15.005077 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d141d12-505e-47f9-ab73-cc904ea7a9a7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:39:15.061758 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:39:15.063338 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0d141d12-505e-47f9-ab73-cc904ea7a9a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e64644dc881469e907a6f6669999757'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:39:15.071969 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0d141d12-505e-47f9-ab73-cc904ea7a9a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e64644dc881469e907a6f6669999757'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:39:15.072716 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0d141d12-505e-47f9-ab73-cc904ea7a9a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e64644dc881469e907a6f6669999757'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:39:15.073327 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0d141d12-505e-47f9-ab73-cc904ea7a9a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e64644dc881469e907a6f6669999757'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:39:15.085349 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0d141d12-505e-47f9-ab73-cc904ea7a9a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e64644dc881469e907a6f6669999757'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:39:15.098208 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1448/2883] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:39:14 2022] DELETE /identity/v3/users/2e64644dc881469e907a6f6669999757 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:39:15.271455 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62ae32-9f6c-477b-b12f-a9c2a666e914 service placement] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:39:15.284181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a62ae32-9f6c-477b-b12f-a9c2a666e914 service placement] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:39:15.331900 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=100 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:39:15.344612 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af4453-5122-46fe-94ff-22b995b7f98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:39:15.345260 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af4453-5122-46fe-94ff-22b995b7f98e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:39:15.345578 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af4453-5122-46fe-94ff-22b995b7f98e admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:39:15.345865 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5af4453-5122-46fe-94ff-22b995b7f98e admin admin] PATH_INFO: `/v3/projects/651fa66c961b4d07a8fa735e63cdef79` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:39:15.347027 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af4453-5122-46fe-94ff-22b995b7f98e admin admin] RBAC: Authorizing `identity:delete_project(project_id=651fa66c961b4d07a8fa735e63cdef79)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:39:15.348971 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5af4453-5122-46fe-94ff-22b995b7f98e admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:39:15.395138 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e5af4453-5122-46fe-94ff-22b995b7f98e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '651fa66c961b4d07a8fa735e63cdef79', 'request_id': 'req-e5af4453-5122-46fe-94ff-22b995b7f98e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:39:15.403790 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e5af4453-5122-46fe-94ff-22b995b7f98e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '651fa66c961b4d07a8fa735e63cdef79', 'request_id': 'req-e5af4453-5122-46fe-94ff-22b995b7f98e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:39:15.409766 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e5af4453-5122-46fe-94ff-22b995b7f98e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '651fa66c961b4d07a8fa735e63cdef79', 'request_id': 'req-e5af4453-5122-46fe-94ff-22b995b7f98e'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:39:15.422500 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e5af4453-5122-46fe-94ff-22b995b7f98e admin admin] The token cache is being invalidate because project 651fa66c961b4d07a8fa735e63cdef79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:39:15.422795 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e5af4453-5122-46fe-94ff-22b995b7f98e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:39:15.423498 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e5af4453-5122-46fe-94ff-22b995b7f98e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:39:15.425596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1436/2884] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:39:15 2022] DELETE /identity/v3/projects/651fa66c961b4d07a8fa735e63cdef79 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:39:15.430009 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d141d12-505e-47f9-ab73-cc904ea7a9a7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:39:15.432489 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0d141d12-505e-47f9-ab73-cc904ea7a9a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:39:15.445978 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d141d12-505e-47f9-ab73-cc904ea7a9a7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:39:15.501958 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ca828-ec94-4cbd-9a65-79c197f3e988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:39:15.502574 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ca828-ec94-4cbd-9a65-79c197f3e988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:39:15.502878 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ca828-ec94-4cbd-9a65-79c197f3e988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:39:15.503147 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664ca828-ec94-4cbd-9a65-79c197f3e988 admin admin] PATH_INFO: `/v3/users/51c3e51644fc46ab9befd6eee1accda1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:39:15.504706 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ca828-ec94-4cbd-9a65-79c197f3e988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51c3e51644fc46ab9befd6eee1accda1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:39:15.506679 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664ca828-ec94-4cbd-9a65-79c197f3e988 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:39:15.550957 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-664ca828-ec94-4cbd-9a65-79c197f3e988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51c3e51644fc46ab9befd6eee1accda1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:39:15.557854 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-664ca828-ec94-4cbd-9a65-79c197f3e988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51c3e51644fc46ab9befd6eee1accda1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:39:15.558507 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-664ca828-ec94-4cbd-9a65-79c197f3e988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51c3e51644fc46ab9befd6eee1accda1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:39:15.559108 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-664ca828-ec94-4cbd-9a65-79c197f3e988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51c3e51644fc46ab9befd6eee1accda1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:39:15.570822 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-664ca828-ec94-4cbd-9a65-79c197f3e988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51c3e51644fc46ab9befd6eee1accda1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:39:15.583137 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1449/2885] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:39:15 2022] DELETE /identity/v3/users/51c3e51644fc46ab9befd6eee1accda1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:39:15.760022 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af4453-5122-46fe-94ff-22b995b7f98e admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:39:15.761441 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e5af4453-5122-46fe-94ff-22b995b7f98e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:39:15.775872 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5af4453-5122-46fe-94ff-22b995b7f98e admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:39:15.834632 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3309c0-23b7-49a7-94de-888318a35c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:39:15.835281 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3309c0-23b7-49a7-94de-888318a35c96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:39:15.835562 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3309c0-23b7-49a7-94de-888318a35c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:39:15.835836 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3309c0-23b7-49a7-94de-888318a35c96 admin admin] PATH_INFO: `/v3/projects/f26452741ee2411aa990bc713efc2886` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:39:15.838136 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3309c0-23b7-49a7-94de-888318a35c96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f26452741ee2411aa990bc713efc2886)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:39:15.840041 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3309c0-23b7-49a7-94de-888318a35c96 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:39:15.887110 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3f3309c0-23b7-49a7-94de-888318a35c96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f26452741ee2411aa990bc713efc2886', 'request_id': 'req-3f3309c0-23b7-49a7-94de-888318a35c96'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:39:15.894497 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3f3309c0-23b7-49a7-94de-888318a35c96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f26452741ee2411aa990bc713efc2886', 'request_id': 'req-3f3309c0-23b7-49a7-94de-888318a35c96'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:39:15.901360 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3f3309c0-23b7-49a7-94de-888318a35c96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f26452741ee2411aa990bc713efc2886', 'request_id': 'req-3f3309c0-23b7-49a7-94de-888318a35c96'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:39:15.912011 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3f3309c0-23b7-49a7-94de-888318a35c96 admin admin] The token cache is being invalidate because project f26452741ee2411aa990bc713efc2886 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:39:15.912293 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3f3309c0-23b7-49a7-94de-888318a35c96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:39:15.912924 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-3f3309c0-23b7-49a7-94de-888318a35c96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:39:15.914440 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1437/2886] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:39:15 2022] DELETE /identity/v3/projects/f26452741ee2411aa990bc713efc2886 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:39:25.579762 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=66,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:39:25.909049 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=80,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:40:15.313875 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ca828-ec94-4cbd-9a65-79c197f3e988 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:40:15.315290 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-664ca828-ec94-4cbd-9a65-79c197f3e988 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:40:15.374190 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664ca828-ec94-4cbd-9a65-79c197f3e988 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:40:15.424152 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20fb857-2a90-4346-9ba1-a66a183b3539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '09b96431ac98447283960019cadde8a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:40:15.424693 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fb857-2a90-4346-9ba1-a66a183b3539 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:40:15.424975 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fb857-2a90-4346-9ba1-a66a183b3539 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:40:15.425247 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20fb857-2a90-4346-9ba1-a66a183b3539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:40:15.427072 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b20fb857-2a90-4346-9ba1-a66a183b3539 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:40:15.491259 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20fb857-2a90-4346-9ba1-a66a183b3539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:40:15.492877 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20fb857-2a90-4346-9ba1-a66a183b3539 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:40:15.502902 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1450/2887] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:40:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:40:21.649167 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3309c0-23b7-49a7-94de-888318a35c96 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:40:21.651090 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-3f3309c0-23b7-49a7-94de-888318a35c96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:40:21.709270 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20fb857-2a90-4346-9ba1-a66a183b3539 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:40:21.710860 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b20fb857-2a90-4346-9ba1-a66a183b3539 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:40:21.725814 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20fb857-2a90-4346-9ba1-a66a183b3539 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:40:21.733224 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3309c0-23b7-49a7-94de-888318a35c96 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:40:21.733811 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966405e-9f8a-4ca2-b66e-dc1e47c134f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:40:21.734536 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966405e-9f8a-4ca2-b66e-dc1e47c134f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:40:21.734894 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966405e-9f8a-4ca2-b66e-dc1e47c134f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:40:21.735245 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966405e-9f8a-4ca2-b66e-dc1e47c134f2 admin admin] PATH_INFO: `/v3/users/32eadc02f9224eb28303aab25506ef2a` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:40:21.736479 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966405e-9f8a-4ca2-b66e-dc1e47c134f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32eadc02f9224eb28303aab25506ef2a)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:40:21.738387 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e20e15-55a7-4cb2-b4db-b8d83951b7a3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe003cd7bfbe4b92b1d64d300c3e7d8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:40:21.738576 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966405e-9f8a-4ca2-b66e-dc1e47c134f2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:40:21.739002 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e20e15-55a7-4cb2-b4db-b8d83951b7a3 service placement] REQUEST_METHOD: `GET` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:40:21.739290 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e20e15-55a7-4cb2-b4db-b8d83951b7a3 service placement] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:40:21.739570 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e20e15-55a7-4cb2-b4db-b8d83951b7a3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:40:21.741336 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-c2e20e15-55a7-4cb2-b4db-b8d83951b7a3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:40:21.754084 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e20e15-55a7-4cb2-b4db-b8d83951b7a3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:40:21.757128 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e20e15-55a7-4cb2-b4db-b8d83951b7a3 service placement] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:40:21.768883 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1438/2888] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:40:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:40:21.788875 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3966405e-9f8a-4ca2-b66e-dc1e47c134f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32eadc02f9224eb28303aab25506ef2a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:40:21.796401 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3966405e-9f8a-4ca2-b66e-dc1e47c134f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32eadc02f9224eb28303aab25506ef2a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:40:21.797358 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3966405e-9f8a-4ca2-b66e-dc1e47c134f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32eadc02f9224eb28303aab25506ef2a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:40:21.798066 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3966405e-9f8a-4ca2-b66e-dc1e47c134f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32eadc02f9224eb28303aab25506ef2a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:40:21.803016 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:40:21.812618 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3966405e-9f8a-4ca2-b66e-dc1e47c134f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32eadc02f9224eb28303aab25506ef2a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:40:21.826365 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1451/2889] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:40:21 2022] DELETE /identity/v3/users/32eadc02f9224eb28303aab25506ef2a => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:40:21.999335 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e20e15-55a7-4cb2-b4db-b8d83951b7a3 service placement] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:40:22.011458 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e20e15-55a7-4cb2-b4db-b8d83951b7a3 service placement] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:40:22.069217 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6290f-b43d-4679-a861-ecae9ea5ddd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:40:22.069896 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6290f-b43d-4679-a861-ecae9ea5ddd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:40:22.070180 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6290f-b43d-4679-a861-ecae9ea5ddd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:40:22.070452 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b6290f-b43d-4679-a861-ecae9ea5ddd3 admin admin] PATH_INFO: `/v3/projects/862a3f47ebe842fb8c5fbd4ef08f0b5f` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:40:22.072640 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6290f-b43d-4679-a861-ecae9ea5ddd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=862a3f47ebe842fb8c5fbd4ef08f0b5f)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:40:22.074596 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b6290f-b43d-4679-a861-ecae9ea5ddd3 admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:40:22.120042 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e6b6290f-b43d-4679-a861-ecae9ea5ddd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '862a3f47ebe842fb8c5fbd4ef08f0b5f', 'request_id': 'req-e6b6290f-b43d-4679-a861-ecae9ea5ddd3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:40:22.127267 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e6b6290f-b43d-4679-a861-ecae9ea5ddd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '862a3f47ebe842fb8c5fbd4ef08f0b5f', 'request_id': 'req-e6b6290f-b43d-4679-a861-ecae9ea5ddd3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:40:22.133517 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e6b6290f-b43d-4679-a861-ecae9ea5ddd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '862a3f47ebe842fb8c5fbd4ef08f0b5f', 'request_id': 'req-e6b6290f-b43d-4679-a861-ecae9ea5ddd3'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:40:22.145172 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e6b6290f-b43d-4679-a861-ecae9ea5ddd3 admin admin] The token cache is being invalidate because project 862a3f47ebe842fb8c5fbd4ef08f0b5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:40:22.145465 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e6b6290f-b43d-4679-a861-ecae9ea5ddd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:40:22.146236 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-e6b6290f-b43d-4679-a861-ecae9ea5ddd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:40:22.147944 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1439/2890] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:40:21 2022] DELETE /identity/v3/projects/862a3f47ebe842fb8c5fbd4ef08f0b5f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:40:22.152398 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966405e-9f8a-4ca2-b66e-dc1e47c134f2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:40:22.154725 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3966405e-9f8a-4ca2-b66e-dc1e47c134f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:40:22.168649 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966405e-9f8a-4ca2-b66e-dc1e47c134f2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:40:22.227767 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebc1f0-5c1e-419b-9129-86e07ce4ac0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:40:22.228441 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebc1f0-5c1e-419b-9129-86e07ce4ac0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:40:22.228725 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebc1f0-5c1e-419b-9129-86e07ce4ac0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:40:22.229001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cebc1f0-5c1e-419b-9129-86e07ce4ac0b admin admin] PATH_INFO: `/v3/users/11e44c7d3e7e4d9c83dd92f4d3757212` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:40:22.230458 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cebc1f0-5c1e-419b-9129-86e07ce4ac0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=11e44c7d3e7e4d9c83dd92f4d3757212)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:40:22.232487 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cebc1f0-5c1e-419b-9129-86e07ce4ac0b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:40:22.283761 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5cebc1f0-5c1e-419b-9129-86e07ce4ac0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11e44c7d3e7e4d9c83dd92f4d3757212'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:40:22.291332 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5cebc1f0-5c1e-419b-9129-86e07ce4ac0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11e44c7d3e7e4d9c83dd92f4d3757212'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:40:22.292077 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5cebc1f0-5c1e-419b-9129-86e07ce4ac0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11e44c7d3e7e4d9c83dd92f4d3757212'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:40:22.292685 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5cebc1f0-5c1e-419b-9129-86e07ce4ac0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11e44c7d3e7e4d9c83dd92f4d3757212'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:40:22.307759 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5cebc1f0-5c1e-419b-9129-86e07ce4ac0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11e44c7d3e7e4d9c83dd92f4d3757212'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:40:22.321984 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1452/2891] 10.222.0.39 () {58 vars in 1224 bytes} [Thu Jul 28 22:40:22 2022] DELETE /identity/v3/users/11e44c7d3e7e4d9c83dd92f4d3757212 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 28 22:40:22.500024 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6290f-b43d-4679-a861-ecae9ea5ddd3 admin admin] Authenticating user token {{(pid=80362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:40:22.501850 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.fernet_utils [None req-e6b6290f-b43d-4679-a861-ecae9ea5ddd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:40:22.518330 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b6290f-b43d-4679-a861-ecae9ea5ddd3 admin admin] Validating token access rules against request {{(pid=80362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:40:22.537255 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:40:22.579547 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25add74-366c-4ec2-80c2-4d242acc77ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b20c2888f9642efb7858178880ed9a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '26d9130b7556481bb766f1b06a63e1e5', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:40:22.580181 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25add74-366c-4ec2-80c2-4d242acc77ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:40:22.580465 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25add74-366c-4ec2-80c2-4d242acc77ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:40:22.580741 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25add74-366c-4ec2-80c2-4d242acc77ad admin admin] PATH_INFO: `/v3/projects/be22185cb3a1487787eaaa875566eebe` {{(pid=80362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:40:22.581993 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25add74-366c-4ec2-80c2-4d242acc77ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=be22185cb3a1487787eaaa875566eebe)` {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:40:22.583930 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25add74-366c-4ec2-80c2-4d242acc77ad admin admin] RBAC: Authorization granted {{(pid=80362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:40:22.630891 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b25add74-366c-4ec2-80c2-4d242acc77ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be22185cb3a1487787eaaa875566eebe', 'request_id': 'req-b25add74-366c-4ec2-80c2-4d242acc77ad'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:40:22.639183 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b25add74-366c-4ec2-80c2-4d242acc77ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be22185cb3a1487787eaaa875566eebe', 'request_id': 'req-b25add74-366c-4ec2-80c2-4d242acc77ad'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:40:22.644683 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b25add74-366c-4ec2-80c2-4d242acc77ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be22185cb3a1487787eaaa875566eebe', 'request_id': 'req-b25add74-366c-4ec2-80c2-4d242acc77ad'} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:40:22.656162 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b25add74-366c-4ec2-80c2-4d242acc77ad admin admin] The token cache is being invalidate because project be22185cb3a1487787eaaa875566eebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 28 22:40:22.656764 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b25add74-366c-4ec2-80c2-4d242acc77ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:40:22.657091 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG keystone.notifications [None req-b25add74-366c-4ec2-80c2-4d242acc77ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 28 22:40:22.658701 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: [pid: 80362|app: 0|req: 1440/2892] 10.222.0.39 () {58 vars in 1233 bytes} [Thu Jul 28 22:40:22 2022] DELETE /identity/v3/projects/be22185cb3a1487787eaaa875566eebe => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 28 22:40:32.317381 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=13,keystone:SELECT=59,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:40:32.653261 dv-cinder-850939-3-iscsi devstack@keystone.service[80362]: DEBUG dbcounter [-] [80362] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 28 22:41:13.558682 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebc1f0-5c1e-419b-9129-86e07ce4ac0b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 28 22:41:13.560074 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5cebc1f0-5c1e-419b-9129-86e07ce4ac0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:41:13.630236 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cebc1f0-5c1e-419b-9129-86e07ce4ac0b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 28 22:41:13.687829 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e716cd43-e819-46e6-83bf-30511b9986b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fe003cd7bfbe4b92b1d64d300c3e7d8b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b9fb2a1c6ad9453d9daf18d42a6f24ec', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 28 22:41:13.688443 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716cd43-e819-46e6-83bf-30511b9986b9 service placement] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 28 22:41:13.688726 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716cd43-e819-46e6-83bf-30511b9986b9 service placement] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 28 22:41:13.689001 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e716cd43-e819-46e6-83bf-30511b9986b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 28 22:41:13.690774 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e716cd43-e819-46e6-83bf-30511b9986b9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 28 22:41:13.702101 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716cd43-e819-46e6-83bf-30511b9986b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 28 22:41:13.703671 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e716cd43-e819-46e6-83bf-30511b9986b9 service placement] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 28 22:41:13.713704 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1453/2893] 10.222.0.39 () {60 vars in 1365 bytes} [Thu Jul 28 22:41:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 28 22:41:23.709202 dv-cinder-850939-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=58 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}